{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B36hiH-mLREY"
   },
   "source": [
    "# Image Classification with Euclidean and Hyperbolic Neural Network\n",
    "\n",
    "Welcome to our first notebook for the ECCV 2022 Tutorial \"[Hyperbolic Representation Learning for Computer Vision](https://sites.google.com/view/hyperbolic-tutorial-eccv22)\"!\n",
    "\n",
    "**Open notebook:** \n",
    "[![View on Github](https://img.shields.io/static/v1.svg?logo=github&label=Repo&message=View%20On%20Github&color=lightgrey)](https://github.com/MinaGhadimiAtigh/hyperbolic_representation_learning/blob/main/notebooks/1_Simple_NN_in_Euclidean_and_Hyperbolic_space.ipynb)\n",
    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/MinaGhadimiAtigh/hyperbolic_representation_learning/blob/main/notebooks/1_Simple_NN_in_Euclidean_and_Hyperbolic_space.ipynb)  \n",
    "\n",
    "**Author:** Mina Ghadimi Atigh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QxU-tJrfUs79"
   },
   "source": [
    "In this notebook, you will perform a simple image classification task using two Neural Networks, one in Euclidean space and one in the hyperbolic space. The main goal is to go through the training and testing process in Euclidean and hyperbolic spaces and see the similarities and the differences.\n",
    "\n",
    "If you prefer working with standard python scripts, feel free to convert this notebook into a python script. To open this notebook on Google Colab, use the button above. Note that you need to copy this notebook into your own Google Drive to save the notebook and trained models. Otherwise, your progress will be lost when you close the browser tab.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wbGnvJGLVlSe"
   },
   "source": [
    "Let's start with importing the libraries and setting manual seed using `set_seed`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "6lH1r7TrUTDI"
   },
   "outputs": [],
   "source": [
    "## standard libraries\n",
    "import numpy as np\n",
    "import warnings\n",
    "from IPython.display import clear_output\n",
    "\n",
    "## Imports for plotting\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "## PyTorch\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "\n",
    "## PyTorch Torchvision\n",
    "import torchvision\n",
    "\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "h5YtvanB-qeK",
    "outputId": "181e3cdc-01b3-4688-ddbe-abcd22741fe3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using device cuda:0\n"
     ]
    }
   ],
   "source": [
    "# Function for setting the seed\n",
    "def set_seed(seed):\n",
    "    np.random.seed(seed)\n",
    "    torch.manual_seed(seed)\n",
    "    if torch.cuda.is_available():\n",
    "        torch.cuda.manual_seed(seed)\n",
    "        torch.cuda.manual_seed_all(seed)\n",
    "set_seed(42)\n",
    "\n",
    "# Ensure that all operations are deterministic on GPU (if used) for reproducibility\n",
    "torch.backends.cudnn.determinstic = True\n",
    "torch.backends.cudnn.benchmark = False\n",
    "\n",
    "# Fetching the device that will be used throughout this notebook\n",
    "device = torch.device(\"cpu\") if not torch.cuda.is_available() else torch.device(\"cuda:0\")\n",
    "print(\"Using device\", device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NDEwear4N7MA"
   },
   "source": [
    "For the Hyperbolic layers and functions, we're going to use geoopt library in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "QQ_t736yOfaK"
   },
   "outputs": [],
   "source": [
    "!pip install -q git+https://github.com/geoopt/geoopt.git\n",
    "! [ ! -f mobius_linear_example.py ] && wget -q https://raw.githubusercontent.com/geoopt/geoopt/master/examples/mobius_linear_example.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "E5Zt53SnOhGJ"
   },
   "outputs": [],
   "source": [
    "import geoopt\n",
    "from mobius_linear_example import MobiusLinear"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ce1Crta0OrLk"
   },
   "source": [
    "Here, we define the paths which will be used in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "r7Wvksp8eAh7"
   },
   "outputs": [],
   "source": [
    "DATA_PATH = './data'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_fvFgVAaWFtG"
   },
   "source": [
    "Let's start with setting up the dataset. In this notebook, you will work with `MNIST` dataset. MNIST consists of 70000 tiny (28*28) gray scale images of handwritten digits, from zero to nine. The goal is to recognize the digit corresponding to each image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 423,
     "referenced_widgets": [
      "995d015dcf3b452e85742c5eea660064",
      "801007438163403591daf4218a0825cc",
      "c9124614852d46d9a111bec0ec03e509",
      "637a7991c79b49f9928259e386c31e7d",
      "62017fd3a3014c04ad6294d363771cf0",
      "16979721c7444c52a282b318160c9586",
      "002bd74778cb4b8da294b4d5f4923e5a",
      "d79cd089fe6540b091b55c677b62967e",
      "d1051e8fde8f4adf8718b27ddbf7c62a",
      "97074fecae464f3f87e907c4f0cf1884",
      "052d04deed0b4a0eb3fc953ab255e5e4",
      "6f7b6a2d42a64e0aa3a285ccb9a7ab43",
      "9a862696188a46d1aa314af61d6574d6",
      "b581f32916574ee6966434311422bf99",
      "6a98047e1ab944899984c2b1d18d78f4",
      "a596527976494de8b123c198ed6ad481",
      "bd0f6652f5dd4f799cdc5c308bb21bf5",
      "dd67739ea09641b396376108d1d0208b",
      "cf7ba4a0d470417e9f8254d3d6840ea5",
      "2192a5dd584a487a903a60b7837f9d65",
      "9e8e11e458054415a57572a6665fc8a8",
      "8d284174e0d64be6a57e9ab5d78eda11",
      "eecdc0d6ec0e499cbdbce64783ed1123",
      "5c247e41508f4bdb8b7c768bf093accb",
      "1e655534822245e8adb36db4cb115d13",
      "10ba0efce9e5430c902cdcf56bab09e4",
      "f243551123fc4dd6a05a9486200614a5",
      "082b385bad4d4e7281a07f77f0cc9cd4",
      "2153a2fbcd98499cafadc5b413cc25f7",
      "d02055dd894c442280f796d130869884",
      "de94640096be496399872f2e94bec4ad",
      "5331b18b991f46f1815942d08c267b44",
      "08cc32263dfa421fa7040d3ba8d22bd8",
      "550154eb777248e590bb8db09ff7a90c",
      "e490e5a6fbea4c87839b4bdfadc8c52e",
      "e010dc5beaf6408f91b22cbac009a395",
      "9d4dd00ba16b403cb233336aab50b80c",
      "51fae820bd914df5b65bbb2678cc05f8",
      "f6aa342888e54445b4447a9fff11f9dc",
      "89b8e5dd0da84f96965e649613e14c54",
      "816d67b923bd4a53b427ee385943b4df",
      "62b3030b82724d72b2bccc65526778da",
      "daecebdd1d6648da9a1fb55ab0c476de",
      "ed605bbbb43b4fdf9bbf8daa08093341"
     ]
    },
    "id": "lD2bxB21cPen",
    "outputId": "6cd88e75-eaf3-447c-e708-9b31ebeedccd"
   },
   "outputs": [],
   "source": [
    "transform=torchvision.transforms.Compose([\n",
    "    torchvision.transforms.ToTensor(),\n",
    "    torchvision.transforms.Normalize((0.1307,), (0.3081,))\n",
    "    ])\n",
    "\n",
    "# Train dataset - downloading the training dataset. Training dataset is splitted into train and val parts.\n",
    "main_trainset = torchvision.datasets.MNIST(root=DATA_PATH, train = True, download=True, transform=transform)\n",
    "trainset, valset = torch.utils.data.random_split(main_trainset, [10000, 50000], generator=torch.Generator().manual_seed(42))\n",
    "\n",
    "# Test dataset - downloading and loading the testing dataset.\n",
    "testset = torchvision.datasets.MNIST(root=DATA_PATH, train=False, download=True, transform=transform)\n",
    "testset_small, _ = torch.utils.data.random_split(main_trainset, [30000, 30000], generator=torch.Generator().manual_seed(42))\n",
    "\n",
    "# Create dataloaders for the train, val and test sets\n",
    "batch_size = 8\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=2, pin_memory=True)\n",
    "valloader = torch.utils.data.DataLoader(valset, batch_size=batch_size, shuffle=True, num_workers=2)\n",
    "testloader = torch.utils.data.DataLoader(testset_small, batch_size=batch_size, shuffle=False, num_workers=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VwNgF3uqgLu_"
   },
   "source": [
    "Before starting the main task, let's visualize some of the images from the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 130
    },
    "id": "jKnbsT8adh9O",
    "outputId": "a0a10415-134c-4b26-f6bd-bc3c926f4236"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAABxCAYAAAB1PMHSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA3s0lEQVR4nO3deVRUV54H8G/te1EUBQUUO4KIKCgq0pqo7Z60WUwyWUzaLB1jYsyJ2j3TTk/rmDmJMT2dSbpjlk46MU4SzTIuJxpNXBp3RMEFBJF9L/Yqiiqo9c4fTr2xZDdQ9Qru55x3jlY9ivu49d77vbv8LocQQkBRFEVRFOUlXF8XgKIoiqKosYUGHxRFURRFeRUNPiiKoiiK8ioafFAURVEU5VU0+KAoiqIoyqto8EFRFEVRlFfR4IOiKIqiKK+iwQdFURRFUV5Fgw+KoiiKoryKBh8URVEURXnViAUf27dvR0xMDMRiMTIyMpCTkzNSv4qiKIqiKD8yIsHH119/jfXr12Pz5s3Iy8tDamoqFi9ejKamppH4dRRFURRF+RHOSCwsl5GRgenTp+O9994DALhcLkRGRmLt2rX4/e9/3+/Pulwu1NfXQ6FQgMPhDHfRKIqiKIoaAYQQmEwmhIeHg8vtv22DP9y/3GazITc3Fxs3bmRe43K5WLBgAc6dO9djf6vVCqvVyvy/rq4OycnJw10siqIoiqK8oKamBhEREf3uM+zdLi0tLXA6ndBqtR6va7Va6PX6Hvtv3boVAQEBzEYDD4qiKIryXwqFYsB9fD7bZePGjTAajcxWU1Pj6yJRFEVRFHWHBjNkYti7XTQaDXg8HhobGz1eb2xsRGhoaI/9RSIRRCLRcBeDoiiKoiiWGvaWD6FQiPT0dBw7dox5zeVy4dixY8jMzBzuX0dRFEVRlJ8Z9pYPAFi/fj1WrlyJadOmYcaMGXjnnXdgNpvxzDPPjMSvoyiKoijKj4xI8PHoo4+iubkZmzZtgl6vR1paGg4fPtxjECpFURRFUWPPiOT5+Dk6OjoQEBDg62JQFEVRFHUHjEYjlEplv/v4fLYLRVEURVFjCw0+KIqiKIryqhEZ8zEauOcp354i1ul0+qI4FDUsuFyuxxx8l8sFlvW8UiOMfgdGDx6P1+/7bK5bGnz0QqFQICYmBlFRUVi8eDGTh6Smpgbvv/8+2trafFxCiho6iUSCVatWITk5GU6nE1arFV988QVyc3N9XTTKC+RyOaRSKZYvX44pU6Ywrx88eBCnT59m/t/V1YWuri5fFJEagoCAALz00kuIiYnp9X2Hw4EdO3bgwoUL3i3YINHgoxdSqRTR0dGYMmUKnn76aSZVbF5eHj7//HMafIxSfWXlY+uTw1AJhUIsXboUixYtgs1mg8ViwdmzZ2nwMQoMlFGSw+FAKpVCpVJh4cKFWL58OYCbT8aNjY24evUqsy8hhAYfLMfhcCCXy/HAAw9gxowZve7T3d2NM2fO4OLFi6y8htHgoxc6nQ4rVqxAZGQkzb46RsTGxmL58uUQi8Uer7e2tuLAgQNob29HV1cXXC6Xj0o4vPh8PgQCwYArT1LsJpfLMW/ePKhUqn7343K5GD9+PLRaLSZNmsS8zuFwsHDhQoSFhTGvFRYW4urVq6iurkZJSclIFZ26Q4GBgXj22WeRmJiIyMjIPvfj8/lYsWIFpkyZgkOHDuHKlSswm83o7u72Ymn7RoOPXmi1WixdurTXKb+DyVlP+R+dToennnqqx/SwsrIy5OTkMKsvj4bgg8PhgMfjgc/n0++zn5PJZLj77rsHXEGUx+Nh5syZPW5WHA4HM2fOxMyZM5nXsrKyIJFIAIAGHywUEBCAp556CqmpqQD6bpnl8/m45557sGjRIrS0tKCyshJ2u50GH2zA5XLB4/EwadIk3HvvveDzb/45xo0b12uLh0wmQ1paGlQqFW7cuMGaShwuarUaEokEAQEBkEgk0Ov1MBgM0Gq1CAoK6vElt9vtsNvtaGtr63XFYjYKDQ1FYmIiQkNDkZSUxAzYioyMhFar7dHyERcXh9WrV8NoNKK7uxttbW3YtWuX3xwvcHP9pNmzZyM2NtbjCXesio6OxpIlSyCTyaBQKGCxWFBaWoqWlhZcvHjRr7ochEIhEhISEB8f3+9+XC53wLwLbjExMZg/fz7a29tx/PhxVjbZj0VBQUF44oknEB8f75Gw02az4cKFC2hqakJ5eTm6u7vx0EMPYcKECQBu1r27dWv//v04e/YsHA6HzydPjPngQygUYsqUKfjDH/4AoVDIvNfbE6FUKkVaWhoUCgWqqqpGVfDB4XCgVqsRFBSEyMhIBAYG4sqVK7Db7YiLi0NCQgIIIR5P/t3d3TCbzSgvL0djY6NfXKS0Wi1mz56N1NRUPPjgg0zA2ZeAgAD85je/Yf5fVVWF48eP+13wMX/+fKSlpfW6uONYExUVheeffx4hISEIDw9Hc3MzfvrpJxQXF6OgoMDvgo/4+HhMnDhx2D4zOjoa0dHRuHz58rB9JvXzBQUF4aWXXsL48eM9XrfZbDh79iwKCgpw/PhxGAwGTJw40SP4+OUvf4m5c+eipqYGeXl56OrqosGHL0ydOhXz5s0Dn88Hj8dDWloaeDzegE3QcrkcM2bMgEqlwrFjx2A0Gr1U4uEnFotx3333ISoqCsDN4CMoKIgZlCaRSDBp0iS0tbUhPDwcGo0GhBCPAMNut8Nms+HKlSsghKC1tRW1tbWsDELS0tIwf/586HQ6TJgwAWFhYT2mHA5GQEAAVq5ciZKSEuzbtw8NDQ0jVOLhw+PxoNPpEBsbC6lU6uvi+ExYWBjS09ORmpqK4OBgKBQKcDgcyGQyTJw4EYQQjwcQNgsKCsIDDzyAcePGQaPRDGv3mfuzpk2bht/+9re4dOkSjh07xsrz+nYpKSlYtGjRoMcyEUJgt9vR3t7OjO1iO3f9uFwudHV1wWAw4MaNG7h27RpMJhNsNhv27duHiooKLF26FBMmTACHwwGHw8HcuXMhkUhw6tQpXLx4ETabDXa73TcHQljGaDQSACO6rV27ltjtduJ0OonT6SQul4u4XK4By+ZwOIjJZCI5OTkkPj5+xMs5kptarSZHjx5l/ga3/y3cW2+v3b5lZWWR1atXk7lz5xIul+vzY+tte/HFF4ndbu9R9qFy/1xtbS3JyMjw+XENZtNoNCQrK6vHMXd1dZHHHnvM5+Xz1paRkUG++uorcvr0aWKz2Xp8F3Jzc0lkZKTPyzmYLTk5mRQXFzPn50hwn//vv/8+4fF4Pj/mgTYul0ueeeYZ0t3d3eO61tdmt9uJwWAgFy5cIImJiT4/hv62xMREUlRUxNSPw+Eger2eXL58uddrkVgsJl999VWPa5fT6SRbtmwhoaGhRC6Xj0hZjUbjgN+vMdXykZaWhoyMDMyePRtcLndMjPR3t2CoVCqEh4czYxqkUil0Ot2Af4PBPFEFBwcjPT0d3d3dOHXq1LCUeyS4o/9b1dTU4NSpU8yAUvJ/T3disRjh4eFQqVSYPHkyMwDP/fP+NlCzt2MfKxISEjB37lyMGzcOCQkJCAwM7LXVSywWIz4+HjweD3V1db57IhwEd30OdP46nU5UVFTAZDIhJiYGSqUS586dQ0VFBTIyMpCYmDjg7xhuXC4XISEhkMvlmDJlCoKCgoblczkcDjIzM8Hn84d0bRcKhQgODsZjjz2GpqYmAIDZbMaPP/7I/J+NLBYLDh06hJKSEjQ2NvZ43+Fw4OjRo+jo6MDdd9/NdMMAN1u1fv3rX+PMmTM4c+aMN4vNGFPBx6JFi/D666/fUXO7v5LL5bjvvvuYC7BarQaAQV24BisqKgoBAQEwmUx+93e9fv06tmzZgvb2drS3tzPBR1BQEObMmYPExETExcUxwQflf6ZPn4533nkHIpGo3xuqTCbDlClTEBAQgNbWVlYHH4PlcDiQl5eHyspKLFu2DDKZDN999x327t2L119/vd/gY6QIBALEx8cjMjIS//Iv/4KUlJRh++yhXts5HA7EYjGioqKwadMm5vzX6/UoLi5mdfBhMpnw0UcfIScnp9dZeO4kY19++SU++OADj+BjyZIlWLx4MV577TUafIwEgUAAgUCA5ORkTJgwAenp6YMa2+FmMBhQWFgIiUSCiRMnDpjKlk0CAgIwb9486HQ6TJ48GaGhoZBKpQMOsHRzuVy4cuUKKisrodVqoVarERwc3OtTikAggFQqZS7ubNbR0YHGxka0tLSgrKwMV69ehcFggMVigdPpZC4+EokE6enpiIuL8/tcLy6XC01NTairq0NQUFCPGT2jVUJCAjIyMjBr1iwIBIIBz1+bzYb6+no0NDTA4XB4qZQjw+l0orGxEQaDAfn5+bhx4wZcLhe0Wi0KCwthMplw6tQpEEKQnp6O5OTkPj9r/PjxePLJJ1FSUoLs7OyfPd1cKBQiPT0dSUlJCAoKGvQ1aSTcer269fvhDw+o5P8mAPRWH1wuF3w+HzNnzkRcXBzi4uKY99wBOCHEp8c4qoMPqVQKuVyORx99FGvXrh1y8FBfX4+dO3ciNDQUsbGxTKZTfxAeHo7XX38d48aNY06kobR0EEKwb98+fPPNN7j77rsxZcoUzJgxA2q1uscXVigUQigU+sVgRr1ej7NnzyInJwc7d+6E1Wrt9Uaj0Wjw2GOPQafT+VXQ2RuHw4HS0lIoFApMnTp1zAQfd911F/7yl79AKBQO6gZnsViQn5+PqqoqWK1WL5Rw5DgcDhQWFqKyshI//PADrly5gr1794LD4cDhcMDlcuHTTz/F559/jrfeeqvf4GPOnDmYNWsW/vu//xsXLlz42cGHVCrFww8/jJkzZ/r9ucVWAoEAMpkML7zwAh566CGfBnh9YV+JhoFUKoVYLMakSZOQkJCA8ePHQygU9hnlGQwG1NXVMRVmNBpRUlKCyspKlJaWghDi82lJg6VSqTBz5kwkJiYiMDBwwNH7TqcTer0eJpMJtbW16OjoAHDzabmoqAgmkwkymQzR0dG9Jl0DbgZpJSUluH79OmtHxJeXl2Pv3r3Q6/UoKipCaWkpuru7weVyERgYCLVajeTkZI9cL3K5vMdJ29XVhdzcXFRWVsJgMPjgSO6M+ynp1vrhcDhQKpUIDg6GyWQaNVPH4+PjMXnyZEybNg1isXjQNziBQICQkBBYLBbU1dX5fUI5l8sFp9Ppsd3Knevh8uXL2LNnDyZOnNhjGidws0XAnZRuOFitVpw+fRotLS0Qi8UQiUSYPHkyNBoNU+5r166hoaGBuWYPdF2x2+2DChjFYjFCQkKgUCgQGxvrN7ObeiMWi5GZmQm5XI5Lly4xM3V4PB5mzJiB2NhYxMTE9NtyKxKJoFQqmTFv3jQqg4+QkBBotVo8/fTT+Kd/+icIBIJ+m5eqqqqwb98+KBQKxMXF4erVq/jrX/8Ks9nMZITzl/7fmJgYvPPOO4iMjBzUE67D4UBOTg6Ki4uxZ88eXLt2zeN9DoeD6OhoLFiwoM+myIsXL+Ldd99FfX09a4O0Y8eOMc3MTqeTuTArFArEx8dj+vTp2LRpE9O6xeVyez1p29rasG3bNuTn5/tVro/ecDgcREREIDk5GcXFxX5/PG6LFi3Ctm3bIBQKh9TaJ5VKkZqaCoVCMWrGfAyEEIIvv/wS33zzDTZv3ozf/va3I94UbzQa8dprr0EgECAsLAwajQbbtm3D7NmzAdy8Ju3evRsHDx5krjkDrc5qNBrR3Nw8YJASGhqK+fPnIzExEb/5zW/8OvgICAjAunXrUF9fj1deeQUXL14EcDOgeOmll7Bs2bIBu4yVSiUiIiLQ0tLi9fEtoy744HA4iI2NRWpqKqKjo/vtCmhubkZ1dTWuXbuG4uJiSKVStLe3o7y8HCaTiYkEbx0LwHZcLhdSqbTP47Zarbh8+TLz1O5wOJCbm4uamho0Nzejq6sLOp0OKpUKQUFBUCqViImJgUAg6PFZVVVVKCkpQV5eHvR6PYxGI2v/Ti6XCw6Hg8lw6r4pyeVyJveHUqkcsOvI5XKho6MDRqPR78cFAGBmfbG9f7s/QqEQIpEI4eHhiI6OxqRJkyCTyXoEHo2NjcjPz4dAIIBarYZCoUBERATzRC+RSJCcnAyBQIBz587BZDL54nD6JZFIEB0djcTExH5vLE6nE7W1tSgrK4PZbO73M20226DyPQQGBmLixIk/O58PIQTd3d2w2Wxoa2uDy+VCdnY2U06Hw4EbN26gpaXFY3xCf7/PbDYPeJxu0dHRHvV+q66uLly6dAlVVVWsy+PkdDqZa49MJmNaLru6uiCTySCVShEcHAy1Wg2NRgOZTDbgZ7q7431x/o/K4ONXv/oVnn/++QGjvkuXLuHDDz9EbW0tCgoKQAgBj8djlhsfjYxGIzZt2oScnBzmNbvdDqfTCZvNBi6Xi/nz52PmzJmYNm0axo0b12cLyuHDh7FlyxZ0dXXBZDKxNvAA/r958Z577sFrr73GXHjcJx+fzx/UjBaXywWTycTqQGusUalUCAkJwYoVK7Bq1SqIxeJeL6Y5OTlYvXo1AgMDcddddyE5ORnPPvss810IDg7Gk08+iaKiIuzduxctLS3ePpQBBQcH47nnnkNcXFy/i8nZbDYcP34cp0+fRnNz87D87qSkJKxevRoXLlzAzp07f3bLkMvlQmtrK9rb27FlyxaPYKCrq2tInz/YczE0NBRPPvkktFptr60eLS0t+P3vf88swsYmNpsNZWVl4PF4SEpKglQqRUBAAJxOJzQaDcLCwrB48WLExcUxySPZbNQFH8DNvrD+BofW19ejoqICV65cQW1tLZqbm9Hd3T0qbiadnZ24cOECqqurwePxwOVymZtqZ2cnmpubUV9f32O8ApfLRUJCAkJCQpCcnIyYmBhotdpeL3A1NTWoqqpCUVER2traWLFOQF+CgoIQFhYGpVIJjUaDCRMmIDAw8I77r93903w+H0VFRejs7BzmElODFRMTA51OB7VaDbVajfj4eAQGBvYIPJqamlBaWor8/Hy0traCy+Uyq3vees67XC6mFYCt1wKRSASdTofw8PB+v8OEEJjNZphMpmHrPuLz+ZDJZMM6+8vdDTrS51FAQACSkpKQmpoKpVLZ44HK5XLBYrGgo6MDBoOBGfvGJlarFUVFRbDb7YiOjmZaP4RCIdNil5SUhIiICMhkMrhcLlRUVDDXaJfLhcjISAQHBzOzv+x2OywWi0+6GEdl8DGQgwcPYtOmTejq6mJy3LP1YjNU5eXlWLVqFYRCIWQyGSQSCWJjY8HhcFBQUACj0dhrc6JQKMT69euxbNkySKVSCIXCXrtaAOC7777Dm2++CYvFwvoWolmzZuHXv/41dDod4uPjhzQAsTfBwcH485//jLq6OjzzzDPIy8sbxtJSg8XhcLBy5Uq8+OKLTNdRXy1XJ0+exLp16zy6UntjsVhQUFCA4uJi1n6vZTIZpk+fjsjIyD7PT+DmTd1kMsFgMPj9wNnhkJqaio8//hgajabXgfM2mw0VFRWoqKhg7cDrlpYWvPvuu4iJicHUqVOZAbpyuRwbNmyAw+GAUCgEj8eDUCiE3W7HJ598gkOHDjELY65fvx4PP/ww1Go1lEoljEYjqqqqfHL/G1PBh16vR11dHW7cuDGohdD4fD4zXddfsqE6HA7m6c4968e9dLper/doSnQvrR4TE4Pg4GDExMQgJCSk1z5AQgjq6+uh1+tRWlrK6uQ7t1IoFIiKikJISMig1sDo7OxEWVkZM55DIpEgPj6eedrj8XgICgqC1Wrt9+LPJoQQdHV1obOzk7UtVP1xt+BFREQgMDAQwM3v7u2re97OZDKhtbUVNTU1aGxsZJ7uHA4HTCYTzGZzj9k/fD6fOV/YRCqVIiYmBklJSZDL5X22PtjtdpSXl6O+vh5Go3HMBx7uGYxBQUEICQnp0ZJrtVrR0NAAk8mEa9euoaamhrXBh3u8WXNzMwoKCuByuRAfHw+ZTNZrQOV0OtHe3o6GhgZ0dHTAarWitLQUV69eRUBAABQKBerq6nx2TRhTwceePXuwbds2dHR0DCrSUygUSEtLw8SJE/3mRuPmcrlgNpuZ8RjAzej+VkKhEEqlEn/84x8xb948qNXqfoOsL774Atu3b2dlk2RfNBoNUlJSBt3NUlhYiBdeeAGtra0AgAkTJuDjjz/2iz7UvjgcDlRWVkIoFCItLc3XxRkymUwGmUyGDRs2YNmyZczr/Y15AICCggJ89913uHbtmscF1mg04tKlS5BIJB6Dht3p1e12O+tmQSQkJOCdd95BREREv+nIDQYDNm7ciJycHFaOWfE2tVqNKVOmYOLEib1eA5qamvDee++hqqoKeXl5TLcLmzU1NWHdunWIiIjAxx9/3Oc5TQhBZ2cnWltbmdlCu3fvxvfff88k2/TloNpRGXy4czncrrS0FNXV1YP+HKlUiri4OERGRnoMUPQX7v7UWy+8HA6HeXIKCQlBUFAQYmNjERkZ2eNn3VtzczPa2tpQVlaGmpoabx/Gz2I0GlFRUQG1Wu3xlOyeams2m6HX65lg9MaNG6iurkZbWxuAmy0fpaWlcLlcCA8PZ25KPB4PWq0WOp0OLS0trG2mB24ea0tLCzOf343D4UCj0SAqKgq1tbU+LGHfOBwOwsPDERYWhtjY2CEFgQ6HA11dXZBIJEhKSkJ3dzc6Ojogk8kQERGB4OBgj2Db3X8+0NR8XxCJRIiIiEBYWFi/3YbuvD11dXVeLB17yWQyjBs3rsc6Vi6XC3a7HZ2dnaiurkZlZSUaGhrQ1dXlw9IOjruOCSGoqalBcHAwNBpNr61ht1//Ozo6WPPwOOqCD5fLhY8++ghff/11j/fcN5TBioqKwquvvgqtVssM7vH3BelEIhFmz56N6OhoLF++HHFxcQgNDe11X6vVCpvNhg8//BC7du3yyyep/fv34+zZs3jiiSfwb//2b8yFu7u7G+3t7Th79ixef/115qLT1dXl8TRQW1uLf/7nf0ZCQgK2bt2KmJgYADdbxZ5//nksWLAA27dvR3FxsdePbbBsNhtOnz6NoqIiLF++nHmdz+fj4YcfxqJFi7Bx40aUlJT4sJS9EwgEWL16NX71q18hODh4SD+rVquRkpKC+fPnIzk5GRUVFfjpp58QERGBe+65ByqVCnK5fIRKPrx4PB6kUikkEgnrAiM2i4+PxyuvvAK1Wu0xyNRms6Gurg7l5eW4fPkyamtrWdvd0heTyYTPPvsMJ06cwAsvvICEhARfF2lIRl3wAdxslhqOMQkikQhhYWFMM6fL5fLpvOg7weVyIZfLmfVXZDIZ4uPjER0djXHjxiE2NtZjf0II2traYDabYbFY0N3djdLSUty4ccNHR/DzuBeMKy0tRWVlJRN8WCwWpjXnxo0bfT7xWK1WVFRUgM/ne3Rb8fl8REZGgsPhsH7ROffgQy6X6zGqncPhMLlcBpMTwBc4HA60Wi3i4+OH/LMSiQQhISGIiIhAYmIi+Hw+ysrKEBER4TGOx83pdDIDstkyNkYgEECpVDIztHz54GO322E2m1ndyucmkUgQGBjIzAy6/ftttVpRU1OD6upqdHR0+EWLR2+cTiccDodfTpgYlcHHSGFzs2xfFAoF7rvvPkRFRWHevHkIDg6GUqmESCRiVri9lcPhwCeffIIjR47AZDLBYrGgoaHBByUfXocOHUJ+fj7zf3e3y0AzINyJfTo6OjwG7/H5fCQkJCAgIMBvnp7HmvDwcKhUKibTaWRkJB555JE+Z3K1trbi66+/RmlpKWuapqOiorBixQokJCT4PEBsbGzEyZMnUVFRwfqBrOnp6Vi7di2ioqJ67Y6orq7G5s2bPbpY/Y1cLsfTTz+NqVOnIiQkxNfFGbIxHXxIJBIolco+A4nbF1FzZ9sTCoXQaDR3nAGREDLi0Tafz4dKpUJwcDDi4+MRGxuLyZMn99l07X46NpvNTE4Ek8nUbxklEgnkcjmsViszc4CtF6XW1lZmEOlQ8Pl8BAYGQqVSefS1u7M0uqdqUyPHYDBAr9czKye7Z7+4uc/L24nFYo+mdrFY3GcXI3DzyV6v16OpqYk12WvFYjEiIyMRGho6Youwuf9OAy3H0NXVhcbGRr9IsBcQEICJEyf2GETvdDrR3d2NtrY2lJSU+OWDFY/Hg0qlQlhYGGJiYhAdHd3rfhwOByqVClqtlpWtO2M6+Fi4cCF+97vf9XlSK5XKXpOVRUREYPv27XfcR+h0OvH2229j7969d/TzgxEZGYlt27Yx02glEkmfC8MBNy+8O3fuRHZ2NnJyctDe3j7gTXXRokV49tlncfr0aXzzzTcwmUx++xTRl+joaGzbtg1xcXHQ6XTM60ajEf/xH/+B/Px8Vo/38Hc2mw1vv/02vvjiCzzyyCOYPn06wsPDERQUxAyIFolEEIlEftMayTbz5s1DZmYmZsyY0e9+bW1tuHLlCiwWC2sfMtwkEgnT3XLr96K5uRnHjx9HUVER627GgxUcHMysRDxu3Lg+9xMIBFi/fj2eeOIJvPXWWzhw4IAXSzmwMRl8iEQiZibLzJkzh5ztUiqVYsqUKUP+ve58C11dXQNOExwqDofDZK1zj1VJT09HXFzcoMvmXuvGveDaQGJjYzF9+nS0trYiNDS0z7EP7qcN9whzf8Dj8aBQKBAWFoZp06b1eLqw2WwoKirCpUuXWNNEPxgOhwNWqxV8Pt8j6JZKpVCpVLBYLD2mZPsSIQQlJSUoLy/HpEmTEBoaCi6X67FKr0QiGdQiim7uBHz+PHB8uHA4HISGhiIlJWXAAb02mw1Go5HV57C7bgMCAiCRSHpMme7q6kJ5eTmqq6tZ07o1FO78TZMnT0ZqaqrHey6XC52dnXA4HJDL5RAKhUhISGDyN7nXyGGLId11t27dij179uD69euQSCT4xS9+gW3btnksw9zd3Y0NGzZg9+7dsFqtWLx4Md5///1+kwF52z333IMXX3wRERERXr0AOZ1OfPHFFzh58qTH2irDQaVSIT09HZGRkVi2bBlCQ0P7bWK+nUAgwJNPPomlS5fCarUO6gKj0+kQGBiIhQsXIikpCXa7vdcbV3V1NQ4cOICGhgbk5eWx6ubWl/j4eGzZsoU5cW/nnkNvMpn8ptvF4XCgoKAAMpkMEydOZG42PB4PTz31FGbNmoXPPvsMP/30k49L2pPT6cTBgwdx7tw5SCQSiEQi5kLK5XKH1CUxY8YM/OEPf/D5GAq2iI2NRWZmJhQKhd+3Hs2bNw+vvvoqdDpdr+N6WlpasH//ftTX1/tdywefz4dWq/WY8n8ri8WCN954AwUFBdi4cSNmzZrFvCcUCiGVSmG1WlkTdA0p+Dhx4gTWrFmD6dOnw+Fw4F//9V+xaNEiFBYWMifyunXrcPDgQXz77bcICAjAyy+/jOXLl+PMmTMjcgBDwePxwOPxEBsbi/nz5w974EEIYXLou7dbV2N0OBwoLCzEmTNn7mj8QX/EYjFiYmKQmJiIuXPnMpkgB8u9tsudTNeKiIhAREREn+9fv34d169fB4fDwZUrV4b8+d7A5XKZgcQ8Hg+hoaG4++67ER4e7rGfy+VCd3c3Ojs7Bx2ksYXL5UJjYyOqqqo8ZjlxOBwkJiZCp9Phxx9/9GEJ+1dTUzMseWYEAkGvF2D3+euvsweGyr0asFqtHnAsTHd3N6tnufD5fAgEAkRHR2P+/Pk9Ag+Hw8FMr6+srPTLtAF8Ph9BQUEICgryaK13t6gbDAZcuHAB2dnZeO655zx+ViAQQCgUMt9vNhhS8HH48GGP/+/YsQMhISHIzc3F3XffDaPRiL///e/46quv8Mtf/hIA8Nlnn2HChAnIzs7GzJkzh6/kdyAlJQXTpk3D9OnTR+TzrVYrTp06hYaGBpSWlsJgMKC5uRkWiwXAzS9JUVERmpqahv3pPyQkBI8//jizqBCb6HQ6PPXUU7h48SKysrJY+cSh0+mwdOlSaLVaTJo0CSEhIb0GcC0tLXjjjTdQVFSEsrIyH5T0ztlsNnz//ffIyclBVFSUR1eSTCaDQCAY1kXD/E1nZydycnJQWVnJyu/ocOJyuXjyyScxf/58TJ06td99jx49ir/97W+oqqpizY3rdhkZGVi+fDlSUlJ6fagsKCjAn/70J1RVVflVN+mtNBoN1q9fj8TERISFhTGvd3R0YNu2bbh8+bLHjD4393T1xMREVFZWorGx0ZvF7tPPGvPhTsbknrKZm5sLu92OBQsWMPskJSUhKioK586d6zX4sFqtHhH1SH4xQkJCkJaWBp1ON6TmRfe0zIFmc5jNZpSVlaGsrAx5eXlobGxETU2NV77sUqkUycnJQ+pq8RaFQoFJkybBaDQOeXyNQCDo9WLS15OpO6Pf7TMg+ksO5853kZaWhtjYWMydO7fHGAJ3C5bZbEZWVhZrW3D643Q6UVpa2mNVY/d4IS6Xe8er/fqbW1sk3d8Td+IpvV7vy6L1cGu24cHo79rmXr9GIBAgLS0N99xzz4B5aiorK/H999+zuntRp9Nh7ty50Gg0vWYyra+vxw8//MD61Ol9cWemzsjIQFJSEoD/z2BtNptx6tQpnD59GsDNe8HtlEoltFotq9bkuuMrjcvlwquvvopZs2YhJSUFwM205kKhsMdgSq1W2+cJvXXrVmzZsuVOizEkMTExWLBgwZC7JPR6Pfbv34+WlhaUl5f32WrhcDhQVVXFzPpwT8Wk7oxcLscrr7yC5ORkj9fd4y16m2104MABHDt2DDqdjkkCxuVyERcXh7S0tB4XZpFIBIVCAZVKhYSEBCYh2+1sNhuqq6tZveolNThWqxV6vR52ux1qtXrEprAOh+7ubtTX10MqlQ5qhgmXy0VAQACz4unteDweHnvsMcyYMQNTp06FVCpl9fEPllwuR1RUVI9AqqioCF9++WW/iQTZTiwWM0khb22ZtFgsOHToEMrKyvoNmLlcLhYuXIjk5GS89957qKio8EaxB3THwceaNWtQUFDARFt3auPGjVi/fj3z/46Ojh7rjAwXtVqN8ePHD7rVwx1ZutNwV1VV4cKFC6Pq5nPrrIGRNtQnJ6FQiAULFmDevHkerxNC0NLS4rFCr/v1kpISZGVlQa1WIz4+ngk+pk+fjqVLl/Zo/ZBIJD3W+Lj9M51OJ6xWKxobG5mb1mjlHrzpre+ELzgcDnR0dDBZMNnM4XCgvb0dJpNpUMEHh8OBTCaDUqns9X2BQIA5c+bgwQcfHPCz/Kn+RSIRAgMDewRSjY2N+OGHH9DS0uK3561AIGAmENzaMmm1WnH58mUUFhb2u0Acl8tFUlISEhISRjS9w1DdUfDx8ssv48CBAzh58qTHQMPQ0FDYbDYYDAaP1o/GxsY+uwPcc/TZqLi4GB999BHq6upw+fJlmEwmv/0C96W7uxuffPJJr32Fw02v1w9bF5RcLu91euXjjz+O1NRUqNVqjyRxISEh0Gg0PQLPgZZP1+v12LFjBxoaGlBTUwOj0Yjm5uZhOQa24XK5ePTRR5GSkoL/+Z//YeWsl+EgEAiYdV3YPt22tbUVx48fh8FgwCOPPDLg/nK5HKtWrepzQCWXyx3SysZD6e5hI6vViubmZhgMBtbnJumLRqPBypUrERcX5xEs2+12Zrr/nSa89KUhBR+EEKxduxZ79+5FVlZWj3VB0tPTIRAIcOzYMTz00EMAbt7Aq6urkZmZOXylHmHuk62hoQE7d+70m8RZd5Jh1Gq14scff8TBgwdHqFQ/j/uYbh3D0d96KrNnz8bs2bN/1u+79WLb1taGffv2oaKiAm1tbazu9x6q28c8cDgczJo1C7/4xS9QUlIyaoMPHo8HmUzmF4u0dXZ2Ij8/H3K5HDabrUed3U4ikWDhwoU/63e6f8etM/bYrLcMt+7z2GazoaOjgxn0748UCgXmzZvHLGrp5nQ6UVtbi8rKSo+/we1rj7nrj22B5JCCjzVr1uCrr77C/v37oVAomH4md0KXgIAAPPfcc1i/fj3UajWUSiXWrl2LzMxMn890GQqr1QqDwYDW1lZWVVZ/KisrsWXLll4HG/XHZrOhsLBwhEr181gsFnzwwQc4evQonnrqKUyYMGHEf2d9fT0OHDjAtHC4E691dnb67ZNTb9zThcVica9jXEYzLpcLkUjkV8ftzk8RGxuLu+66a8jn+VDY7XZYrVacPXsWhw8fxtWrV1n73Z8zZw6WLVvWYzzX1atX8d1336GoqIjVU4R/DrFYjDlz5iA6OhoTJkxgMlgLBAJmHCZwM4g8cuQIrly5gsuXL/uotD0NKfj44IMPAABz5871eP2zzz7D008/DQD4r//6L3C5XDz00EMeScb8id1uR2trK4xGI2tPuts1NDTgo48+8nUxhlV3dze+++47KJVKzJ49mxnlPZKam5uxZ88e1NbWoqSkZNR1swE3n4CsViu6urogEAj86iY8HNyze/zpuA0GA7KysqDX6zFt2jSm5W84W27cD1p2ux0WiwXnz5/Hu+++y+oHsGnTpmHdunU9us9KSkrw4YcforOzc1Sew8DNMXHp6ekYN24c7r33Xo/lH9zcLcdnz57Fnj17WLWWzZC7XQYiFouxfft2bN++/Y4LNdzcycUGO6q7qqoKf//731FWVjaqBpf6K6vVii+++ALZ2dker3M4HERERCAoKAgJCQn9Jjpz6+zsREtLC2pra5GXl9ejG6WxsRFlZWXo6OgYVV0st3I6ncjOzobNZsO8efPuKLEc5V1msxkFBQVob2+HVqtFREQE5syZ0+fA0qFyN8m7XC6cOXMGP/30Ey5evMjqwKM/LpcLNpuNtXlJhoNQKERKSgqsVmuvy2E4HA78+OOPKCwsxNmzZ9HU1MSqGT9jYlI/j8eDQCAYdA6DmpoafPrpp36bjGa0sVqt2LVrV4/XORwOMjIykJiYiCVLlgxqaXu9Xo/i4mJkZ2fj008/HbVPRf1xOp04f/48amtr7zirLeVdFosF165dQ01NDex2O8aPH4/09PRhCz6A/89nlJ2djT//+c/D9rm+MBaCD4FA0CMNwa2cTicOHTqEffv2ob29nXXjXsZE8DEQs9mMtrY21NTUIDs7G9euXRu1/YSjCSEE9fX1sNvtcDgcg5qx09HR4bGA3ljkcrmYZeNvna7scrlw/Phx5OXl4cKFCz4sIdUXq9WK6upqyGSyYQmcCSE4c+YMrl69yiR8ZMNSGAMJCgqCRqNBcHBwv11PQqEQCoUCdrsdJpPJL1tympub8be//Q3x8fF44IEHEBQUBKBnl5vdbkd+fj6am5tRVVWF1tZWXLp0ibVdTzT4AGAymVBeXo6TJ0/ijTfeoF0tfqS6uhrV1dXIzc0dVP+3P158hps7aGttbfWYokcIwb59+/D+++/TvxNLWa1WlJWVQSwWD8sSDYQQHDlyBB9//DFMJhPMZrNf1L1Wq0VKSkqPtZduJxKJoNVqYbFY0NnZ6RfHdruGhga8+eabSExMxKxZs5jg43Z2ux1nz55Ffn4+jh49ioqKClYf75gIPtzpti9evIgPP/ywx/smkwl6vR4lJSWjuplutGPzicZGTqcTR48eZVJOE0JQUFAwZv+OYrEY8fHx4PF4qKurY+XToltbWxt27drFrEx8pwghuHjxIsxmM+x2u9/UfVtbG8rKyvpcoHPcuHF4/vnnwePxEBgYiPLycnz99dd++2BJCEF7ezt2797d5wrxdrsdZ86cQX19PYxGI/vrkrCM0WgkAEZk43A4hMfj9di4XC7hcrmEw+GM2O+mG93YuHG5XI9zYSycA0uWLCEGg6HHtaeyspKsW7eO3H///UShUPi8nEOtuzvd/LXOORwO2bBhA3E6nT3q0ul0EofDwWyHDx8mKpXK52UeyTp338PYUJ9Go3HAe/2YaPlwI/+XKpuiqJv8ZSr5cKqvr8fu3bsRGxuLOXPmMBmW3eMp3IM62W4s1t2tSD9P9u6ptzU1NTh9+jQuXbo07CuJ+8JoqvMxFXxQFEXl5+fjlVdewZw5c5Cens4EH2azGbm5uaiurh5VF/mx7MqVK3j55ZfR0dFBu9RZhgYfFEWNKeT/0m67s9m6cyRUVFSMuky2o11JSQn27t2L+Ph4pKamoq2tjclI3NLSgpycHFgsFhp4sBCH9Nd25QMdHR1MmliKoqiRwuVyPRYndLlcsFqt7B+oRzH4fD6EQiFWrVqF//zP/0ROTg527dqF0tJSnDlzhpk+THmX0WgcMAcNbfmgKGpMcrlcrEu8RA2Nw+GAw+FAaWkpfvrpJ1y/fh1lZWWor6+nrVgsR1s+KIqiKL8mFAohlUrhcDhgs9ngcrloV4sP0ZYPiqIoatSz2WyjYjbLWMIdeBeKoiiKoqjhQ4MPiqIoiqK8igYfFEVRFEV5FeuCD5aNf6UoiqIoaggGcx9nXfBx6yqbFEVRFEX5l8Hcx1k31dblcqG4uBjJycmoqakZcLoO5V0dHR2IjIykdcMytF7Yi9YNO9F6GX6EEJhMJoSHhzPr6/SFdVNtuVwudDodAECpVNIvBUvRumEnWi/sReuGnWi9DK/B5uliXbcLRVEURVGjGw0+KIqiKIryKlYGHyKRCJs3b2aWuqbYg9YNO9F6YS9aN+xE68W3WDfglKIoiqKo0Y2VLR8URVEURY1eNPigKIqiKMqraPBBURRFUZRX0eCDoiiKoiivosEHRVEURVFexcrgY/v27YiJiYFYLEZGRgZycnJ8XaQx5d///d/B4XA8tqSkJOb97u5urFmzBkFBQZDL5XjooYfQ2NjowxKPXidPnsSyZcsQHh4ODoeDffv2ebxPCMGmTZsQFhYGiUSCBQsWoKSkxGOftrY2rFixAkqlEiqVCs899xw6Ozu9eBSjz0D18vTTT/c4h5YsWeKxD62X4bd161ZMnz4dCoUCISEheOCBB1BcXOyxz2CuX9XV1bj33nshlUoREhKC3/3ud3A4HN48lFGPdcHH119/jfXr12Pz5s3Iy8tDamoqFi9ejKamJl8XbUyZOHEiGhoamO306dPMe+vWrcP333+Pb7/9FidOnEB9fT2WL1/uw9KOXmazGampqdi+fXuv77/11lv4y1/+gg8//BDnz5+HTCbD4sWL0d3dzeyzYsUKXLt2DUeOHMGBAwdw8uRJrFq1yluHMCoNVC8AsGTJEo9zaNeuXR7v03oZfidOnMCaNWuQnZ2NI0eOwG63Y9GiRTCbzcw+A12/nE4n7r33XthsNpw9exaff/45duzYgU2bNvnikEYvwjIzZswga9asYf7vdDpJeHg42bp1qw9LNbZs3ryZpKam9vqewWAgAoGAfPvtt8xrRUVFBAA5d+6cl0o4NgEge/fuZf7vcrlIaGgo+dOf/sS8ZjAYiEgkIrt27SKEEFJYWEgAkAsXLjD7HDp0iHA4HFJXV+e1so9mt9cLIYSsXLmS3H///X3+DK0X72hqaiIAyIkTJwghg7t+/fDDD4TL5RK9Xs/s88EHHxClUkmsVqt3D2AUY1XLh81mQ25uLhYsWMC8xuVysWDBApw7d86HJRt7SkpKEB4ejri4OKxYsQLV1dUAgNzcXNjtdo86SkpKQlRUFK0jL6uoqIBer/eoi4CAAGRkZDB1ce7cOahUKkybNo3ZZ8GCBeByuTh//rzXyzyWZGVlISQkBOPHj8eLL76I1tZW5j1aL95hNBoBAGq1GsDgrl/nzp3DpEmToNVqmX0WL16Mjo4OXLt2zYulH91YFXy0tLTA6XR6VDoAaLVa6PV6H5Vq7MnIyMCOHTtw+PBhfPDBB6ioqMBdd90Fk8kEvV4PoVAIlUrl8TO0jrzP/ffu73zR6/UICQnxeJ/P50OtVtP6GkFLlizBzp07cezYMWzbtg0nTpzA0qVL4XQ6AdB68QaXy4VXX30Vs2bNQkpKCgAM6vql1+t7Pafc71HDg+/rAlDss3TpUubfkydPRkZGBqKjo/HNN99AIpH4sGQU5R8ee+wx5t+TJk3C5MmTER8fj6ysLMyfP9+HJRs71qxZg4KCAo/xahR7sKrlQ6PRgMfj9Rh53NjYiNDQUB+VilKpVEhMTERpaSlCQ0Nhs9lgMBg89qF15H3uv3d/50toaGiPwdoOhwNtbW20vrwoLi4OGo0GpaWlAGi9jLSXX34ZBw4cwD/+8Q9EREQwrw/m+hUaGtrrOeV+jxoerAo+hEIh0tPTcezYMeY1l8uFY8eOITMz04clG9s6OztRVlaGsLAwpKenQyAQeNRRcXExqquraR15WWxsLEJDQz3qoqOjA+fPn2fqIjMzEwaDAbm5ucw+x48fh8vlQkZGhtfLPFbV1taitbUVYWFhAGi9jBRCCF5++WXs3bsXx48fR2xsrMf7g7l+ZWZmIj8/3yM4PHLkCJRKJZKTk71zIGOBr0e83m737t1EJBKRHTt2kMLCQrJq1SqiUqk8Rh5TI2vDhg0kKyuLVFRUkDNnzpAFCxYQjUZDmpqaCCGErF69mkRFRZHjx4+TixcvkszMTJKZmenjUo9OJpOJXLp0iVy6dIkAIG+//Ta5dOkSqaqqIoQQ8uabbxKVSkX2799Prl69Su6//34SGxtLurq6mM9YsmQJmTJlCjl//jw5ffo0SUhIII8//rivDmlU6K9eTCYT+e1vf0vOnTtHKioqyNGjR8nUqVNJQkIC6e7uZj6D1svwe/HFF0lAQADJysoiDQ0NzGaxWJh9Brp+ORwOkpKSQhYtWkQuX75MDh8+TIKDg8nGjRt9cUijFuuCD0II+etf/0qioqKIUCgkM2bMINnZ2b4u0pjy6KOPkrCwMCIUColOpyOPPvooKS0tZd7v6uoiL730EgkMDCRSqZQ8+OCDpKGhwYclHr3+8Y9/EAA9tpUrVxJCbk63/eMf/0i0Wi0RiURk/vz5pLi42OMzWltbyeOPP07kcjlRKpXkmWeeISaTyQdHM3r0Vy8Wi4UsWrSIBAcHE4FAQKKjo8nzzz/f4wGK1svw661OAJDPPvuM2Wcw16/KykqydOlSIpFIiEajIRs2bCB2u93LRzO6cQghxNutLRRFURRFjV2sGvNBURRFUdToR4MPiqIoiqK8igYfFEVRFEV5FQ0+KIqiKIryKhp8UBRFURTlVTT4oCiKoijKq2jwQVEURVGUV9Hgg6IoiqIor6LBB0VRFEVRXkWDD4qiKIqivIoGHxRFURRFedX/AkVhqmK0PI56AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     2    2    3    1    4    0    7    8   \n"
     ]
    }
   ],
   "source": [
    "# Extract class names\n",
    "classes = torchvision.datasets.MNIST.classes\n",
    "# Class names are like 0 - zero\n",
    "classes_num = [c.split('-')[0] for c in classes]\n",
    "\n",
    "# functions to show an image\n",
    "def imshow(img):\n",
    "    # What should I do here to unnormalize? \n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "# get some random training images\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# show images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('     '+' '.join(f'{classes_num[labels[j]]:4s}' for j in range(batch_size)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LuxtGTdggwxr"
   },
   "source": [
    "## Implementing Neural Networks\n",
    "\n",
    "### Euclidean Neural Network\n",
    "\n",
    "Let's start with defining a simple Euclidean neural network. Our simple neural network consists of three fully connected layers. To create the fully connected layers in Euclidean space, `nn.Linear` in used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "VpibIkf7g0jP"
   },
   "outputs": [],
   "source": [
    "class EuclideanNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.fc1 = nn.Linear(28 * 28, 750)\n",
    "        self.relu1 = nn.ReLU()\n",
    "        self.fc11 = nn.Linear(750, 20)\n",
    "        self.relu11 = nn.ReLU()\n",
    "        self.fc2 = nn.Linear(20, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        out_x = self.fc1(x)\n",
    "        out_x = self.relu1(out_x)\n",
    "        out_x = self.fc11(out_x)\n",
    "        out_x = self.relu11(out_x)\n",
    "        out_x = self.fc2(out_x)\n",
    "        return out_x\n",
    "\n",
    "euclidean_net = EuclideanNet().to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MWoZ_HJlzhAh"
   },
   "source": [
    "### Hyperbolic Neural Network\n",
    "\n",
    "Next step is to create a neural network in hyperbolic space. This network consists of three fully connected layer, similar to the netowrk defined in the Euclidean space. To define a fully connected layer in the hyperbolic space, `MobiusLinear` is used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "2mgCnV3buXNb"
   },
   "outputs": [],
   "source": [
    "def hyperbolic_ReLU(hyperbolic_input, manifold):\n",
    "    euclidean_input = manifold.logmap0(hyperbolic_input)\n",
    "    euclidean_output = F.relu(euclidean_input)\n",
    "    hyperbolic_output = manifold.expmap0(euclidean_output)\n",
    "    return hyperbolic_output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "JcZZwhLgdZ4t"
   },
   "outputs": [],
   "source": [
    "class HyperbolicNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.fc1 = MobiusLinear(28 * 28, 750)\n",
    "        self.fc11 = MobiusLinear(750, 20)\n",
    "        self.fc2 = MobiusLinear(20, 10)\n",
    "\n",
    "    def forward(self, x, manifold):\n",
    "        out_x = self.fc1(x)\n",
    "        out_x = hyperbolic_ReLU(out_x, manifold)\n",
    "        out_x = self.fc11(out_x)\n",
    "        out_x = hyperbolic_ReLU(out_x, manifold)\n",
    "        out_x = self.fc2(out_x)\n",
    "        return out_x\n",
    "\n",
    "hyperbolic_net = HyperbolicNet().to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Yg1n29ZG0Opo"
   },
   "source": [
    "Here you can see the overview of both Euclidean and hyperbolic Neural Networks. The main structure of the networks are similar, one consisting of *Euclidean* Linear layers and the other one of *hyperbolic* layers. \n",
    "\n",
    "As the Hyperbolic Net consists of hyperbolic layers, the input must be in hyperbolic space, too. Therefore, `Exp map` which stands for `Exponential Map` is used to project the input of the Neural Network (flatten image) from Euclidean space to the hyperbolic space. `Exp map` will be performed later in the training and testing loop."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center width=\"90%\"><img src=\"figures/1_net.png\"></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "svfLDiPdhyeF"
   },
   "source": [
    "### Optimizer and Loss function\n",
    "\n",
    "Next step is to define the optimizer and loss function. For training a neural network with hyperbolic layers, the optimizer should be in hyperbolic space, too. Here we use `SGD` and `Riemannian SGD` as the optimizers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "X-lucVWXh3bA"
   },
   "outputs": [],
   "source": [
    "criterion = nn.CrossEntropyLoss()\n",
    "euclidean_optimizer = optim.SGD(euclidean_net.parameters(), lr=0.001, momentum=0.9)\n",
    "hyperbolic_optimizer = geoopt.optim.RiemannianSGD(list(hyperbolic_net.parameters()), lr=0.001, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-LGZoVGyh4K8"
   },
   "source": [
    "Finally, it's time to implement the training loop."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3S-tKiAgT3Vy"
   },
   "source": [
    "### Euclidean training loop\n",
    "\n",
    "First, we start with training the Euclidean neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hb4EkcIqiLob",
    "outputId": "f95681d8-a3bd-445e-e848-b4585ef933ec",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  1250] loss: 0.732\n",
      "Accuracy of the network on the 10000 val images: 89 %\n",
      "[101,  1250] loss: 0.000\n",
      "Accuracy of the network on the 10000 val images: 96 %\n",
      "[201,  1250] loss: 0.000\n",
      "Accuracy of the network on the 10000 val images: 96 %\n",
      "[301,  1250] loss: 0.000\n",
      "Accuracy of the network on the 10000 val images: 96 %\n",
      "[401,  1250] loss: 0.000\n",
      "Accuracy of the network on the 10000 val images: 96 %\n",
      "[501,  1250] loss: 0.000\n",
      "Accuracy of the network on the 10000 val images: 96 %\n",
      "[601,  1250] loss: 0.000\n",
      "Accuracy of the network on the 10000 val images: 96 %\n",
      "[701,  1250] loss: 0.000\n",
      "Accuracy of the network on the 10000 val images: 96 %\n",
      "[801,  1250] loss: 0.000\n",
      "Accuracy of the network on the 10000 val images: 96 %\n",
      "[901,  1250] loss: 0.000\n",
      "Accuracy of the network on the 10000 val images: 96 %\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(1000):  # loop over the dataset multiple times\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # get the inputs; data is a list of [inputs, labels]\n",
    "        inputs, labels = data\n",
    "        inputs = inputs.to(device)\n",
    "        labels = labels.to(device)\n",
    "\n",
    "        # zero the parameter gradients\n",
    "        euclidean_optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        flatten_inputs = torch.flatten(inputs, start_dim=1)\n",
    "        outputs = euclidean_net(flatten_inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        euclidean_optimizer.step()\n",
    "\n",
    "        # print statistics\n",
    "        running_loss += loss.item()\n",
    "        if (epoch % 100) == 0:\n",
    "            if (i % len(trainloader)) == (len(trainloader) - 1):   \n",
    "                print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / len(trainloader):.3f}')\n",
    "                running_loss = 0.0\n",
    "                # ================================================================================\n",
    "                correct = 0\n",
    "                total = 0\n",
    "                with torch.no_grad():\n",
    "                    for data in valloader:\n",
    "                        val_images, val_labels = data\n",
    "                        val_images = val_images.to(device)\n",
    "                        val_labels = val_labels.to(device)\n",
    "                        # calculate outputs by running images through the network\n",
    "                        flatten_val_images = torch.flatten(val_images, start_dim=1)\n",
    "                        val_outputs = euclidean_net(flatten_val_images)\n",
    "                        # the class with the highest energy is what we choose as prediction\n",
    "                        _, val_predicted = torch.max(val_outputs.data, 1)\n",
    "                        total += val_labels.size(0)\n",
    "                        correct += (val_predicted == val_labels).sum().item()\n",
    "                print(f'Accuracy of the network on the 10000 val images: {100 * correct // total} %')\n",
    "                # ================================================================================\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "48PBvQOYURzF"
   },
   "source": [
    "### Hyperbolic training loop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WzNCI5QyUXAR"
   },
   "source": [
    "Next step is to train the hyperbolic neural network. \n",
    "\n",
    "In addition to the optimizers, there is one more difference between Euclidean and hyperbolic training loops; `Exponential Map`. As the hyperbolic Net expects hyperbolic input, data is projected to the hyperbolic space using `manifold.expmap0(flat_inputs)`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "Nck7RTVCUbPJ",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  1250] loss: 1.975\n",
      "Accuracy of the network on the 10000 val images: 82 %\n",
      "[101,  1250] loss: 1.430\n",
      "Accuracy of the network on the 10000 val images: 95 %\n",
      "[201,  1250] loss: 1.427\n",
      "Accuracy of the network on the 10000 val images: 95 %\n",
      "[301,  1250] loss: 1.426\n",
      "Accuracy of the network on the 10000 val images: 95 %\n",
      "[401,  1250] loss: 1.425\n",
      "Accuracy of the network on the 10000 val images: 95 %\n",
      "[501,  1250] loss: 1.425\n",
      "Accuracy of the network on the 10000 val images: 95 %\n",
      "[601,  1250] loss: 1.424\n",
      "Accuracy of the network on the 10000 val images: 95 %\n",
      "[701,  1250] loss: 1.424\n",
      "Accuracy of the network on the 10000 val images: 95 %\n",
      "[801,  1250] loss: 1.424\n",
      "Accuracy of the network on the 10000 val images: 95 %\n",
      "[901,  1250] loss: 1.424\n",
      "Accuracy of the network on the 10000 val images: 95 %\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "manifold = geoopt.PoincareBall()\n",
    "\n",
    "for epoch in range(1000):  # loop over the dataset multiple times\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # get the inputs; data is a list of [inputs, labels]\n",
    "        inputs, labels = data\n",
    "        inputs = inputs.to(device)\n",
    "        labels = labels.to(device)\n",
    "\n",
    "        # zero the parameter gradients\n",
    "        hyperbolic_optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        # ------------------------------\n",
    "        # Hyperbolic inputs\n",
    "        flat_inputs = torch.flatten(inputs, start_dim=1)\n",
    "        hyp_inputs = manifold.expmap0(flat_inputs)\n",
    "        # ------------------------------\n",
    "        outputs = hyperbolic_net(hyp_inputs, manifold)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        hyperbolic_optimizer.step()\n",
    "        \n",
    "        # print statistics\n",
    "        running_loss += loss.item()\n",
    "        if (epoch % 100) == 0:\n",
    "            # print statistics\n",
    "            if (i % len(trainloader)) == (len(trainloader) - 1):   \n",
    "                print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / len(trainloader):.3f}')\n",
    "                running_loss = 0.0\n",
    "                # ================================================================================\n",
    "                correct = 0\n",
    "                total = 0\n",
    "                with torch.no_grad():\n",
    "                    for data in valloader:\n",
    "                        val_images, val_labels = data\n",
    "\n",
    "                        val_images = val_images.to(device)\n",
    "                        val_labels = val_labels.to(device)\n",
    "\n",
    "                        # calculate outputs by running images through the network\n",
    "                        flatten_val_images = torch.flatten(val_images, start_dim=1)\n",
    "                        hyp_flatten_val_images = manifold.expmap0(flatten_val_images)\n",
    "                        val_outputs = hyperbolic_net(hyp_flatten_val_images, manifold)\n",
    "                        # the class with the highest energy is what we choose as prediction\n",
    "                        _, val_predicted = torch.max(val_outputs.data, 1)\n",
    "                        total += val_labels.size(0)\n",
    "                        correct += (val_predicted == val_labels).sum().item()\n",
    "                print(f'Accuracy of the network on the 10000 val images: {100 * correct // total} %')\n",
    "                # ================================================================================\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "w9OXEPi9iY9_"
   },
   "source": [
    "### Testing function\n",
    "Once the training loop is complete, it's time to see how the trained Euclidean and hyperbolic models work given the testing split.\n",
    "\n",
    "First, we visualize a batch of test images and the predicted labels given the batch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "6Kx06ie0imNZ"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAABxCAYAAAB1PMHSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA1cklEQVR4nO3deXxTdb4//tc52ZumSfd9XyillL2lsojIOg4qLiOCj+uoc72OVS/gqMMdl6s+rssdHZdxuc7oxe0qDCqoKAylSBEoBUqhtEDovqfpmiZt9nx+f/DN+RHTQkvT5KT9PB+P84AkJyef089Z3uezMoQQAoqiKIqiKC9hfZ0AiqIoiqImFxp8UBRFURTlVTT4oCiKoijKq2jwQVEURVGUV9Hgg6IoiqIor6LBB0VRFEVRXkWDD4qiKIqivIoGHxRFURRFeRUNPiiKoiiK8ioafFAURVEU5VXjFny8++67SEpKglQqRV5eHo4fPz5eP0VRFEVRlB8Zl+Bj+/bt2Lx5M5577jmcOnUKM2bMwMqVK6HVasfj5yiKoiiK8iPMeEwsl5eXh3nz5uGdd94BADgcDsTHx+PRRx/FH//4xyt+1+FwoK2tDQqFAgzDeDppFEVRFEWNA0II9Ho9YmJiwLJXLtsQevrHLRYLysrKsGXLFu49lmWxbNkylJSUuK1vNpthNpu5162trcjKyvJ0siiKoiiK8oLm5mbExcVdcR2PV7t0dXXBbrcjMjLS5f3IyEhoNBq39V9++WUolUpuoYEHRVEURfkvhUJx1XV83ttly5Yt0Ol03NLc3OzrJFEURVEUdY1G0mTC49UuYWFhEAgE6OjocHm/o6MDUVFRbutLJBJIJBJPJ4OiKIqiKJ7yeMmHWCzGnDlzUFRUxL3ncDhQVFSE/Px8T/8cRVEURVF+xuMlHwCwefNm3HvvvZg7dy5yc3Px5ptvYmBgAPfdd994/BxFURRFUX5kXIKPu+66C52dnXj22Weh0Wgwc+ZM7N27160RKkVRFEVRk8+4jPMxFv39/VAqlb5OBkVRFEVR10Cn0yEoKOiK64xLyQfFTyKR6KoDv1yOEAJCCOx2OxwOxzimjKIoippMaPAxSYSEhGDz5s1ISkoa0fqEEHR0dKCvrw/79u2jc/NQFEVRHkODj0lAIBAgKCgIq1evxuzZs0f0HUIIampq0N7eDrVaPSmDD4FA4PLabrf7KCWew7LskH3wHQ4HeFYDS1HUBEaDjwkuPDwcDz/8MDIyMpCQkDCq70ZEREAul0OlUo1P4niIZVmIxWIkJibigQceQFBQEGw2G7RaLT744AO38Wv8iUwmw4MPPug2irDD4cDnn3+OI0eO+ChlFEVNNjT4mMAYhoFSqcSdd96JadOmXdN3FQoF5HI5GIaZFE/GDMNALBYjJiYGGzZsQFRUFMxmM6qrq7F9+3a/Dj7EYjFWr16NlStXurxvtVpx4sQJHD16dMLk8WQ5XsfilyVg/vb3ujz9/pZ2igYfE1ZkZCTuv/9+pKWljamLM8MwuOWWWxAfH48ff/zRZfC4iSg8PBwrVqxAZmYmZDIZjEYjLly4gJqaGphMJl8n75qJRCJIJJIhGxyzLIu1a9ciOTkZu3fvRmlpqQ9SOHZBQUFQKpWYNWsWFi9ejNLSUnz99dejbiwdEBCAnJwcMAyDiooKDAwMjFOKvY9hGMycORMJCQlIT09HdHQ0rFYrbDYbCgsLUVxc7Oskjsjy5cuxatUq7vWRI0ewc+dOjwUhEokEubm5kMvlOHHiBLq7uz2yXer/R4OPCSosLAz33Xcf0tPTXd4f6cnpfKpgGAZLlizB9ddfj66urgkffAQHB+OGG25AYmIipFIpjEYj6urqUFdXB4vF4uvkXTOhUDhsbyeWZbF06VLk5+ejsbHRb4MPuVyO6OhoXH/99di4cSM+/PBDfPPNN6Pejkwmw/Tp08GyLKqrqydU8MGyLDIzM5Gbm4ulS5di+vTpMJlMMJlM6O7u9pvgIy8vD5s2beKuU1KpFLt27fJo8DF79myEhoaiurqaBh/jgAYfE4xYLEZ4eDiio6MhEoncPh8YGMC2bdvQ1NQ05PcFAgFuueUWzJw5c5xTyk8BAQFIT09HbGwshEIh+vv7sW/fPtTV1UGv1/s6edckICAA99xzDzIzM5GcnOzr5IybmTNn4rbbbsO0adNGNLHVcORyORYsWIDQ0FCkpaWhvb0d27ZtQ1tbmwdT6xsMwyA9PR3z58/nSkSFQiGkUimEQv7fDpYtW4ZFixZh0aJF47J9pVKJu+++G4mJiZg5cyYYhsH333+PpqYm2Gw2XlXvKBQKrFu3jpu63mQy4R//+Adqa2t9nLKR4f/RRo2KSCRCTEwMoqOjh7yYGAwGbN26FUePHh3y+2KxGMnJyZM2+JDJZEhPT0d4eDiAS4PeFRUVob6+3scpu3YBAQFYv349Fi9e7OukjKucnBw88MADAEY2q+Zw5HI58vPzkZ6ejptuugktLS0oLi6eMMFHamoq8vLyuNdCoZBb+O7GG2/EU089BWBseTwcpVKJf/u3f8OMGTMAAD09PQgNDYVYLIbD4eBVjzeFQoEHHngAubm5AC4N7FVWVkaDD1/Kzs5GYmIiIiIiEBoa6vZ5fX09qqqqoNPp0NnZCYfDMWEG0ZJKpUhOTkZiYqJLyUd/fz927dqFmpoatLa2un1PKBRi9erVyMrKcmmcSgiBWq1GU1OTX9+Ar0YoFCIwMBAKhQIMw4zLhc1bgoKCcPPNNyMiIgIMwyAwMBDx8fFX3CehUAiZTIZly5YhMDAQFy9eRENDAzo7O9HZ2enF1I/dWPPObrejq6sLwcHBUKlUfn0sOLEsi+nTpyMuLg4xMTEu++QPjXOXLl2K2bNnIy8vzyXtJ06cwMGDB1FSUjKmfQgODsatt96KtLQ0REREgBACg8GA3t5eGI1GWK1Wn98jAgMDsXDhQgQHByM8PBxhYWGIjo722+NzwgUfDMNgzpw5WL58OWbNmoXMzEy3dfbu3YtPP/0UDQ0N6Ovrg81m8/mB5SlSqRRTpkxBamoqxGIx975Op8Pbb7+NU6dODXmSikQi3HPPPbjjjjvcDuaKigocOHAAarV63NPvK2KxGKGhoVCpVKMaBZaPgoOD8dhjjyEnJwdCoXBEwZRIJIJIJMKtt96Km266CTt37sS+fftQUVHhd8HHWFmtVmg0GigUCgQGBvo6OR4hEAiQl5eH3NzcUXe59zWGYbBmzRo89thjbsfxoUOH8Mc//nHM1++wsDD84Q9/QGZmJhiGgd1uR29vL7q6ujA4OMiL9l4qlQp33nkn0tPTMWPGDAQGBvpt4AFM0OAjIiIC6enpCA4OHvJGkpCQgOXLl6O7uxu5ubkeK04zm804fvw4urq60Nvb65MDdmBgAGVlZWhqakJ3dzdkMhkAoLe3F1qt9opPBwzDDHvjdQ61PlEFBQVh5syZmDJlCkQiEfr7+1FZWYnz58/DaDT6OnmjJhAIIBAIhh1U7HLOzwkhEAgEYBgGGRkZsNvt3Ei3AwMDMBgM3kj6qEVGRiIqKgrR0dEe2Z5AIEBISAhCQkLcBprzVwzDIC4uDlOnTnUZt4cQgpaWFnR0dKC9vd13CRxGfHw8wsPDERER4XJtKisrw/Hjx3H06NExBR4sy0Imk0Eul3PnCwAYjUYUFhaiuroaWq12zPvhCUKhkDvOxWKx27VaLBbjpptuQnx8PIqKitDQ0OCbhI7QhAs+ACAxMRFz584d9qI7bdo0TJ06lXvtqZtqX18fnnnmGZSXl8NkMvkk+Ojr68OePXsAwOWJlxAyYUp3xkNoaCiWL1+OpKQkiMVitLe34+uvv0ZdXR1vb7pX4gw+rvV78+bNw9y5c2G1WlFbW4u2tjbe/h1SUlKQn5+PtLQ0j2zP2W4qJibGI9vjA4FAgClTpmD+/Plu18Xz58+jtLSUd20FGIbB1KlTua7BToQQ7N27F88999yYr2kCgQBKpRJKpdLlfDEYDPjf//1flJaW8ua66WyPl5qaOuTnMpkMBQUFGBgYwL/8y7/Q4MMXmpqacPLkScTHxyMqKgoajQYdHR2IiIjg6sjG44lGJpNh/vz5CAkJQWdnJ3Q6ncd/YyScwdRIgiqhUIhFixYhJSUFiYmJQ26rs7MTFy9eRE9Pj8fTyhcBAQFISkpCbGwsBAIBTCYT6uvr0dTUBKvV6uvkjUhQUBCWLFmC1NRUBAcHj7pIdqh2ABkZGVi9ejUOHz48bA8pX5FIJJBIJEhJScHcuXO5Vv/nz5/HyZMncezYsTE9WPhzkbaTc7BApVIJmUzm9rTsLPkoLy/n3QB6DMMgMTERc+bM4dphDA4OwmQywWAwjKm0mmVZSKVShIaGYuXKlUhJSYFCoeCqW7RaLcxmMy8Cj+DgYCxZsgRpaWlQKpXDHpfO+5pEIsENN9zAzSrrcDhw7Ngx3gWXEy74IITg5MmTsFqtWLNmDaKioqBWq1FcXIyFCxd6rGh2KAEBAdiwYQN6enpw7Ngx1NXVjdtveYpMJsOjjz6KX/3qV8O2dq+trcWhQ4cmdLWLUqnE3LlzuZu2wWDAyZMn0dLS4jf7HRkZiRdffBGZmZke6bnAMAwWLlyI6667Dq+99hoKCws9kErPUSgUCA4ORm5uLu68807uxlpUVIQnnniCF40EfY1lWcTGxiImJgYKhWLIdSorK/Hdd9/x7m/FMAxmz56N2267jcvbvr4+aDQa9PX1jWnbQqEQwcHByMzMxJYtWxAfHw+hUAiz2YyGhgY0NTXxpro1Li4Or7zyCpKSkoYcPuGXxGIxHn74YS4/bTYbCgoKaPAx3px11Gq1GkqlEr29vTh79iyqqqrgcDiGLI0IDAxEeHg4BgcHodVqIZPJEBkZ6faU4HA4YLVaYbFY0NPTA4lEgvnz50OpVLr8vj9M0iUQCBAfH4/IyEiEhIRAIpG4fE4IQVNTE7RaLTQaDe8uTOOBZVmXPHe2eeA7kUiE8PBwxMbGQi6XuzQ0vhKHwwGtVgu9Xg+j0QiLxYKEhARERERw6zirYZzdTuvr63Hu3Lnx2pVRycjIwKxZs5Ceng6BQICWlhbU19fj4sWLMJvNfpF3400gECA7OxtZWVkICwsbch2+dSFlWRazZ89GUlISUlNTuUCaEILa2lqUlpaO+cFOoVAgPz8fU6ZMgUKh4G7qJpMJp06dQnV1NW/G9XFO+TDS85phGJcghWVZXrZdmnDBB3Cp2FWtVuPAgQNgWZY7ub777rshMyE9PR2LFy9GU1MTDh48iOjoaCxfvtwts61WK3p7e9HX14fS0lKEh4dj+/btXPDhLBYcGBjg1ck8FLFYjKVLl2Lq1KlD1m07HA4cPnwYhw4dQmVlpQ9SSI2UXC5Hbm4u0tLSuAbGI+FwOHDmzBmo1Wq0tLSgr68P69atw9KlS93WXb58OebMmYNPPvkEL7zwgs9v7AzD4Ne//jX+/d//HSKRCAzDoKSkBFu3bkVDQ4PP08cXEokEv/nNb/CrX/1qRE/NfCAUCvHQQw9h3bp1btfgwsJCvPbaa7DZbGP6jZiYGGzatAnx8fEuDXD7+/vx0Ucf4cyZM7zo4TKRTcjgwzlux0gP0K6uLtTW1kKr1cJgMKCnpwe1tbVuJ6vNZoNer4fVakVUVBRiYmJcTg6LxYJTp06hsbERvb29Ht0nTxMIBIiLi0NaWhrkcvmQ63R1daG+vt5nbVe8QaVSIS0tDVOnTvWLQZaGIpPJkJWVhdTUVEil0quub7PZcPbsWWg0GpSVlaG5uRmdnZ3Q6/U4fvw4bDYb0tPTXUZDlUgkUKlUmDJlClasWIHm5macP3/eJzf5mJgYhIWFISYmBgEBAdz7BoMBbW1t6O/vv6btKhQKrsfT5dv1d2Kx2O24IISgrq4OGo2Gl71cJBLJkNclq9U6pjmWVCoVZs6ciczMTISHh0OhUIBlWQwMDOD06dOoq6tDV1cXzGbzWJLvEVKpFHFxcUhOTna5FxmNRpSXl3MlMyKRCFOmTIFSqRxypFqGYTBt2jQsX74cFy5cQHNzs1f3Y1iEZ3Q6HQHg1YVlWSIWi4lQKHR5LZFIXBaxWExEIhGJj48nzz//PPn8889JV1cXl/aOjg6ycuVKEhQUxG2Lr0twcDDZs2cPMRqNxGazueWD1WolDz/8MBGLxYRlWZ+nd7yW+fPnk++++46UlZURi8XC7f/x48dJTEyMz9M3kiU7O5tUVVWRwcFBYrfbr3qOGQwGsmHDBqJSqYhcLicymYw7vgMDA0lwcDB5++23Xb7jcDiI3W4nRqOR9PT0kLfeeosIBAKv7yvDMGT9+vXkb3/7Gzlz5oxLGl999VUiFAqv+XidPn06qaioIDqdzuWcaG5uJnPmzPF5Pl/LolAoyPfff+92DNhsNvLnP/+ZLFy4kHfHuVgsJp999plbmh0OB3nqqafGtO358+eTixcvEp1OR6xWK7Hb7cThcJCamhqSm5tLgoKCfHJcD7XEx8eTp59+mmzdupX09fVxf4fGxkaycOFColKpiEqlIklJSeT//u//yPnz54lerx/y72YwGEhXVxf53e9+55W063S6q16H/PNRz8McDodLEdsvX/8SwzCIjY1FXFwcxGIx7HY7N1pqb2/vNT95eYNIJEJmZibi4uIQHh7u9kTkcDhQU1MDjUaDlpaWCV/0KJVKERUVhdDQULAsC7PZjJ6eHm7kW75jWRZCoRABAQHDVrlYrVZcuHCBK8EymUxoamoastGeM7+NRiPsdjvXXdu5SKVSSKVSBAQEgGVZr1cvMgyDkJAQJCYmIigoiGtj5WyPNZbieKFQCKVSyfUScA421tTUxIsn4dFwjmwbEhIybHWLTqeDRqPB4OCgl1N37ZKSkrBgwQLudX9//xV74QUEBCAsLIxry5WTk4OwsDAuj53sdjv6+/t5de0WCARQKBRc6YzNZkNvby80Gg16enq489dut6OyshIMwyAoKMhtYDyGYSCXyyGVSt3a9vkSDT6uQUBAAHJzc5Geng6xWAyTyYSysjI0NDTwvopCpVLhpZdeQl5enktDWSebzYa3334bO3bs4NWJOF7kcjnS0tK4E7ynpwf79u3DhQsXxlS86w0sy3LdTa/ULbS/vx9PP/00jh07BgAghFw1b202GywWy7BzfjiDHl80VkxLS8PixYu5dFksFq7BrCf19/fjq6++Qk1NDbq6ujy67fEmFAoxdepUxMXFubRpcCKEoLu7Gw0NDbxvn3a5e+65B2vXruVeHzt2DEVFRcNW/2VkZODWW2/lqsdFItGQ1z0+EggEXDDhrBo6cuQIamtrXRrDGgwGvPPOOwgLC8OHH37oN+PT0OBjFGQyGVJTU13q14BLF+q2tjY0NTX5xQ0rJCSEmzjNyeFwoLGxEVqtFg0NDbwZ1W+8iEQiyGQyBAYGQiQScTcyo9GI2tpabhZLvrvS0OkWiwV1dXVobW1Fc3PzqPK0ra0N5eXliI+P58bP4AuRSMSde4QQtLe3o6Gh4ZonfpPL5UhJSUFmZqZLKYHNZkNLSwsaGxv9ruRDIBAgISEB6enpw3axtdvtvDzGnXmqVqsRGRnJBU/O0pzLn+zj4+ORnp4+bPCRlJSEyMjIYXuKGI1G1NXVobq6mnfXbmd3YKVSyfXEk8lkXKmjEyEEer0eLMv6zZhEAA0+RiUlJQV/+9vfkJiY6HLzdg7FW1FR4XdPSE4WiwWvv/46fvzxx0kxl0dISAiysrK4bppOHR0d+PLLL7khxf1Zb28vnnzySZw6dWrUx+Xnn3+Ob7/9Fps2bcKmTZvGKYWe8c033+DNN9+85q6RU6ZMwQcffIDY2FiX83pwcBCHDx9GVVUVb8Z8GCmpVIq77roLixcv9psnfSe73Y7t27ejtLQUDz74IFasWDHsullZWUhJSRn2c6FQeMVePo2NjSgoKOAa3/JJYGAg5s6di5iYGK6Ec+HChYiLi8N7773n6+SNGQ0+RoBlWQQEBEClUnGzQgKXnox6enrQ3t4OrVaL7u5u3kaeQqEQcXFxiIuLG7JtACEEXV1daG5u9qti2GulUCiQlpbmNsOnzWaDTqfjTR//a2GxWNDS0oLm5mY0NDQMOYvx1eh0Ouh0umGrZ5w9X7q7u30+EFt/f/817aOTWCxGbGys2wCEDoeD6zrvbxiGgUql4toyOTlLFS5vM8A35P+NqkwIQXV1NeLj4yGTySCRSNwm+3O2QRrOwMAA6uvrIZVKERkZ6TbUgtVqRWtrK9ra2nh53bt80lOWZSGXyyGXy7mJIFUqFcRiMQICAhAcHHzViRAvry71dZs2GnyMgEKhwKxZs5CVleXSYEen0+Hzzz9HTU0Nzp07B61Wy8sDGLg0a+Prr7+OGTNmDFkn6Gy4x8di2PGQlZWFp556ijt5JxKtVotNmzahsrLymqsirubGG29EdnY2vv/+e2zZsoW3Qfdk5ayO++VAiVarFW+++SZ27drF26pVZ4DU2dmJl156CX/961+5LtDLli3DokWLRrytiooKvPTSS8jKysLTTz/tVgVFCIHdbufldVuv1+PkyZPo6urC7NmzuYdGlmURGBiImJgY3HnnnUhKSsLs2bO57udXEhgYiNDQUOj1ep83NKbBxwhIpVIkJCQgNjbWpfGd1WpFU1MT6uvrMTAwwMsbN8uyUCgUCA8PR2pqqtukRA6Hg+ulw9eJw8aDTCZDTEwMd0LbbDaYTCYMDg76zQBVYrEYcXFxiI6OdjkubTYbGhoaxjQKpEqlQlBQ0LBF9s7GrkKhcELMgXI58v9mcHYu/sbZG8k5Q/EvaTQaVFdX+yBlI2e1WmG1WrnqLuessykpKYiPjx/xdpztOZRKpcuTvtlshlarRVtbG28DZ7PZjKamJgiFQkyfPp27VgmFQsTGxoJlWaSmpiI5ORkZGRkIDQ0FcOma3tXVBZPJhNDQUG68FIZhEBoaiuTkZDQ2NtLgwx/ExMSgoKAAsbGxLl20TCYTTpw4gaqqKt7euIOCgrB27VqkpaVxB+flBgcH8V//9V8oLi5GY2OjD1LIDz09PSgtLUVFRYXfdC9OS0vDm2++yU077kn33HMPNmzYMGxj06KiIrzyyivQarW8vXhfK4fDAaPRiMHBQZ8XTY+WWCxGTk6O28id/u7ChQtobGzEsWPHhm1AOxSDwYD29nakpqa65GVNTQ02bdrETSHBR+3t7XjvvfeQnZ2NBQsWcPeesLAwvPjii7BardzUGJdXt5jNZrz22msoKSnBM888w7WZYRgG69evx8qVK/GXv/wFW7du9cl+OdHgYwQkEgni4+O5OmG73Q6DwYDe3l709vbyunutSCRCUlISkpOTh6wbdTgcqKurQ0VFhQ9Sxx+Dg4NobGzkbd3vUAICApCZmYnY2FiPbdNZlxwfH48ZM2YMO+prT08PF6j5unRAoVAgKioKNpvtioGQQCDghmK/XGhoqFtbAL7NdzJSLMsiODgYYWFhfjOc+kgMDg5icHAQ3d3do/qeRCJBcHCwy1gZRqMRnZ2dqKys5OXork7OtlsqlQp9fX1QqVSQyWQQi8VIS0tzW9/hcKCvrw99fX24cOECzp496zLSNsMwiI6O5sY18jUafFyDrq4ufPrpp6itreV9z5CAgAAsXrwYWVlZftfq3ZtaWlrw0UcfQavV+l3PBk/KyMhAUlISUlJSIBaLh61ScbYP4sMNet26dVi4cCEaGhpQU1Mz7HoRERGYNm2aW6ARGBjoUkpw+cBl/kYoFCIpKQmZmZnDTpswmcybNw/PP/88oqKiIJfLodVq8eOPP0KtVvu82mGkenp6sGPHDqSkpGDNmjUICQkZcr2BgQG88MILKCkpQW1tLSwWy7DHMB8antLg4wqEQiECAwMRFBTk0nDLbDbj4sWLqKmp4V3fcCeBQMA1LoqKinKZqRS4dPNwlt5MtGLzKxGLxZDL5QgMDHS5sRqNRjQ0NPC2B8B4k8vlkEgkiImJQXJyMlQq1ZCTMFosFlgsFp/OGjs4OIje3l7IZDJIpVLEx8cjPj4eISEhkEqlw6YrJiYGc+fOveocPs7Zr3U6HS/bcV2Js+QjNDTUpSE1IYSrSpoM57tQKIREIkF0dDRyc3O5agnned7S0uI3eWs2m1FbW8sNDujs+eO8Jzmv5d3d3aioqMDx48cBXLrWOecqc57fTnK5HKGhoTAYDD7rzTWq4OPll1/GN998gwsXLkAmk+G6667Dq6++iilTpnDrmEwmPP7449i2bRvMZjNWrlyJ9957D5GRkR5P/HibNm0ann32Wbf6U6PRiLNnz6K6upq3T8lxcXF4/vnnkZaWNmS9vcFgwAsvvICTJ0+iqqrKByn0jRtuuAEbN25EbGzshCqWHguBQICHHnoIq1atglKphFwudwtWnSoqKlBYWIgTJ0745InJ4XDgyy+/xIkTJ3DPPfdgzZo13GdxcXFXLN2TSqUjmlq8ra0Nf/rTn1BTUzOmLry+IBaLkZeXh3nz5rk8IVutVuzYsQPl5eU4c+aMD1PoHRkZGVizZg2ys7PdJmW7ePGi3wwiCFzqSv7TTz+hsrISYrEYSUlJWLVqFcLCwrjPn3/+eZSXl+Ps2bPc96xWK9566y189dVX2LhxI1avXs19tn79eixYsACffvopPv30U6/vEzDK4KO4uBgFBQWYN28ebDYb/uM//gMrVqzAuXPnuCK+TZs24YcffsCOHTugVCrxyCOP4LbbbsORI0fGZQfGA8MwEIlEiIiIwPXXX+/SithisWBgYADd3d1XnFPA1wIDA3HdddchPT3d5X2HwwGTyYS+vj6cOHEChw4d8lEKfSM6OhpLly6dcN1rr5VEIoFMJkNOTg6WLVt21fU7Oztx+vRpNDc3+6zko7q6Gm1tbcjPz4der+e6lAoEgivWZTscDgwMDHDpFggEkMlkblVLg4ODOHHiBO97hAxFIBAgIiLCrR0QIQS1tbU4efLkqNtN+COVSoUZM2YgMTHRJeC0Wq1cOz1/qVazWq1oa2vD4OAgzp8/D7PZjNzcXK4ko6+vD8ePH8fRo0ddvkcIQVVVFdRqNe644w7YbDZupNS0tDSkpqbi559/9sUuARhl8LF3716X1x9//DEiIiJQVlaGxYsXQ6fT4aOPPsIXX3yBpUuXAgC2bt2KqVOn4tixY5g/f77nUj6OoqOjsXjxYuTk5LjcpLRaLXbs2IGamhqXhjz+RKfT4ZVXXsHZs2dx7tw5XyeH8iGRSISCggJcf/31mDFjxoi+o9FoUFJSAr1e77Pgw2g0wmq1YuvWrfjpp58QEhKC4OBgpKenIysra9jv1dTUYM+ePVy1w/Tp0/Hkk0+6TTI2ETkcDrS0tODChQu87ZnnSWFhYcjNzYVKpXKpMjeZTGhoaEBHR4fflHw4DQwM4Oeff8apU6dw/PhxrgOBxWKBWq0e9nuEEDQ2NuLUqVNITEzkTS3EmNp8OHt5OIv3ysrKYLVaXZ6gMjMzkZCQgJKSkiGDD7PZ7DJvAh8mM1MoFNywvZfXDw8MDKCsrAz19fW8rW65GrPZjMOHD7tFyZ7AsuwVqzKcM4/yibNXAx8aTnqSQCDgnvicJQO/fMKXSCTIzc3FzTfffNXt2Ww2blbN1tZWnz41OtNSUVGBiooKREVFITo6GgaDYdiZfQHg9OnT+P7777mu1P39/cMej/44dolzEsCh0u6c/2MylHoA7uP4OBtIG41Gvx3B2Gq1orm5GcClrsej0dvbi5aWFoSGhroEH862Mb5oPH7NwYfD4cDGjRuxYMECZGdnA7j0VCQWi936l0dGRg47bv7LL7+M559//lqTMS7Cw8OxYsUKREZGutUXnj9/Hg0NDX430ZQTy7JQKpUICQmBXq/3SDAgEAggFAqxYMEC3HvvvcPWq1+8eBFvvvkmLwJMp5aWFhw6dAhnzpzxm/E9riYgIAA33HADN+eFsx2As47YSSAQIC8vb0Tb3LNnD7Zt2wa1Ws274mqdTgeLxQK9Xn/F9kvd3d0jetoVCoUICwvjutL7wxNyYGAgfv3rXyM1NRVRUVG+Tg7v1NfXY/v27bh48aJfDpc/Fg6HA/v378e5c+fw2GOPuQw0efPNNyM5ORm7d+/Gtm3bvJquaw4+CgoKUFlZicOHD48pAVu2bMHmzZu51/39/aMawc6TnE+ISqUSmZmZXOM15xC8RqORm8fFXzEMA7lcjqCgIJjNZo/cSEQiESQSCTIyMrBu3bph21McOXIEf//73zEwMOCTkoahZoDt6+vDqVOnUFNT4xc3mZEQi8WYMmUK9xAQEBCAW2+9FYmJide8zXPnzuGLL77wUAo9y2g0wmg0oqen54pdbUfq8jk0+DyGz+UkEgk3BcTl1UjObsN2u93n47F4g/Ma/sth5bu6ulBUVIS2trYJ85AxUoQQrnfm2rVrXdp+5OTkICcnBy0tLdi+fbtXj5FrCj4eeeQR7N69G4cOHXLpSREVFQWLxcINiOLU0dExbDTunK2PD2bOnIn7778fqampLsW39fX1+OCDD1BXV+e3bT2cAgMD8eCDD2Lt2rUwmUweueE6G/ulpaVdsTdBamoqXn31VVy4cAHvvfeeV7u1RkREIDExEcnJyW4XpolGJpNh0aJF3DgGIpFo2LEBKHc2mw3d3d0jLinhA6FQiISEBLfBBE0mEz788EOUl5fj1KlTPkyhd8yZMwe33347pk2b5lJqbTKZ0NTUhK6urglXxTpShBDs27cPOp0ON954I6677jruM5FIBKlUetWB+jxpVMEHIQSPPvoodu7ciYMHDyI5Odnl8zlz5kAkEqGoqAi33347AECtVqOpqQn5+fmeS/U4SUpKwn333cf13HFGgR0dHfjiiy+g0Wj85mI0HJlMhuXLl/vktyMjI7FhwwaUlZXhk08+8WrwoVQqkZaWNmQXUn+dw8OZbofD4VKqI5FIMHXqVI9sezJyOBzQ6/V+1S7A2dMnIiLC5aZrtVrxz3/+Ez/88IMPU+c9KSkpWL9+PRQKhcuDkMViQVdX16Qdxwe4dE6fPn0ara2tSEhIcAk+BAIBpFIpTCYTP4OPgoICfPHFF/j222+hUCi4dhxKpRIymQxKpRIPPPAANm/ejJCQEAQFBeHRRx9Ffn4+r3u6OIdd/uWIjjqdDlVVVThz5gxMJtOkjZg9RavVYv/+/bhw4YLXRxfMyMjA+vXrkZCQ4FLy0dvbi9LSUnR0dPhd/jY3N+OFF15ARkYG7r//fo+VbhBC8MMPP6C4uBgOhwOEEJw4ccIj26bGB8MwCAgI4CZgm6zEYjEUCsUVGx5PZjqdDlar1a3H06JFi/Diiy+iuLgYX331lVcePEYVfLz//vsAgCVLlri8v3XrVvz2t78FALzxxhtgWRa33367yyBjfMayLMRisVtPDb1ej4qKCqjVap+O6HgtnGl1/jserfdH+/fo7u7G3r170djY6PXeQgkJCVi9erXbhVmn0+Hs2bN+2f1Qo9Hg73//O3JycnD77bcjODgYwLXl9eV5SQjB4cOH8dZbb8Fut/OugSnljmEYbjbbyUwoFEIul9NxfIZhMBhgMBhcZu9mGAazZ8/G7Nmz4XA48PXXX/Mv+BhJgqRSKd599128++6715wob8vOzsYdd9yBqVOnugQgPT09KCoqQnNzs9/1buns7MTbb7+N1NRUrF+/ftgRK8eirKwMBw8eHPFw211dXThz5oxfDlvNZzqdDv/85z+RkpKC/Pz8Uc36CVw6r511vYWFhSgrK8Phw4cnTSNFauKLjo7G3XffDbvdjqioKO4hxFmlPtmqY/bt2weDwYAlS5ZgyZIl3AOLTCZDWFgYBgcHx/2BjM7tAiArKwuPP/64W7VLT08PDh486Jd94zs7O/HOO+8gNTUVy5YtG7fg44033oDBYOBV99nJpr+/H/v27UNaWhqys7NHHXwAl9oGGI1G7N69Gx9++OE4pJKifCc6Ohq/+c1vEBgYiJycHO4h8+zZs9izZ8+kCz7279+P/fv3g2EYl5oMqVTKdTOnwcc4cg4pPXfuXAgEAi7waGtrw9GjR3HmzBm/K/H4pb6+Pnz88cfjMqrdsWPHYDAY/PJvdOHCBezZs4ebFt6fmUwmVFdXo6enB++//z7Cw8ORmZkJlUqFqVOnugUjdrsdarUa3d3dYBgGhBCo1Wq0traioqLCR3vhe85GtrS0Z+KRyWSIj4+HWCwGy7LQaDTYtWsXqqurJ13gcbmjR4/itddeQ25uLjf7+X333YfS0lJ8880341rlOqmDj9zcXLzyyitus1w2NTXho48+QmtrK29nrR2p7u5uvP7667xo88Enp0+fxp/+9Ce/Han2ckajEZWVlQCAQ4cOQalU4q677kJaWhpiYmKGDD7Ky8tx7tw5sCwLQgh+/PFHnD592q/z1BOcvYcm+99hopHJZNyge8ClwQVfeukltLS0+DBVvldYWIj9+/fjiSeewOLFizFr1izMmjULH374IXbt2kWDD09TKpUIDw9HZGTkkANPAf59Yx3KRNuf0aqqqsIHH3zA5XVZWdmEbHdCCIHZbEZVVRU6OzthtVrdRhy22+0oKyuDRqPhSj46Ozsn7THS39+Pc+fOob+/H42NjWhqavKbwcWcBgYG8PXXX6OyshIrV65EZGQkbDYbLBbLpGow3NbWhgMHDiA2NhZZWVlc246WlhYUFhZypbSNjY1+1ZV6PF1+3juvjyKRCHK5HCaTadxKtidl8BEZGYnZs2cjMTHRL+dwoEbv559/dplZ2fmEOxEZjUZuX3ft2jXkMf7LfZ+sgQdwqXTwu+++Q11dHfbu3Qu9Xu93x0Zvby9effVVREVFIT09HRERETCbzTAajX63L2NRU1ODzz77DPn5+ZgyZQoXfKjVajz55JNcFctEPv89QSKRQKVSob+/nwYfnhQbG4tFixYhPT19yAuzzWaDXq93mX6b8m/OIfIni192tabcaTQabN++HUajEWfPnoVWq4XJZPLbm5LD4YDBYMCePXtQXV0Ni8WCgYEBtLW1+TppXqPX61FbWwuWZREQEMBVqVdWVsJoNE6qa8BoVFZW4rPPPkNmZibmzZuH4OBgTJs2Dc3Nzejr6xuf6wjhGZ1ORwCM6/LII48Qk8lEbDYbcTgcbmnYv38/SU1NJSqVijAMM+7poQtd6OL9hWEYIhKJiEgkIgKBgLAs6/M0eWIRCoVELBYTsVhMRCLRpLuGsSxLBAIB9zcQi8VEKBT6PF18XliWJWKxmGzcuJHYbDZSUVFB3njjDbJhwwYiEAhGvT2dTnfVe/2kLPlwTv3+yzk+WltbcfLkSZSXl8NgMPh9LwiKooZHCPHaUNLeNBHbMo2Gs+SKlnKMnMPhgMVigVqtxs6dO9Ha2orKykq0tLSMW+nppAw+hnPy5En87ne/47qPjtcfnaIoiqL4prCwEMXFxVw1tcPhGLdqyEkZfDQ2NmLfvn1u7T3Kysqg1+v9ctwKiqIoihoLm83mtZIzhvDs8b6/vx9KpXJcf0MikQw58ZDVasXAwMC4/jZFURRFTWQ6nQ5BQUFXXGdSlnyYzWZaukFRFEVRPsJefRWKoiiKoijPocEHRVEURVFeRYMPiqIoiqK8infBB8/av1IURVEUNQojuY/zLvigk/1QFEVRlP8ayX2cd11tHQ4H1Go1srKy0NzcfNXuOpR39ff3Iz4+nuYNz9B84S+aN/xE88XzCCHQ6/WIiYlxG0H8l3jX1ZZlWcTGxgIAgoKC6EHBUzRv+InmC3/RvOEnmi+eNdJxunhX7UJRFEVR1MRGgw+KoiiKoryKl8GHRCLBc889B4lE4uukUL9A84afaL7wF80bfqL54lu8a3BKURRFUdTExsuSD4qiKIqiJi4afFAURVEU5VU0+KAoiqIoyqto8EFRFEVRlFfR4IOiKIqiKK/iZfDx7rvvIikpCVKpFHl5eTh+/LivkzSp/Od//icYhnFZMjMzuc9NJhMKCgoQGhqKwMBA3H777ejo6PBhiieuQ4cOYc2aNYiJiQHDMNi1a5fL54QQPPvss4iOjoZMJsOyZctQXV3tsk5PTw82bNiAoKAgqFQqPPDAAzAYDF7ci4nnavny29/+1u0cWrVqlcs6NF887+WXX8a8efOgUCgQERGBW2+9FWq12mWdkVy/mpqacNNNNyEgIAARERF44oknYLPZvLkrEx7vgo/t27dj8+bNeO6553Dq1CnMmDEDK1euhFar9XXSJpVp06ahvb2dWw4fPsx9tmnTJnz//ffYsWMHiouL0dbWhttuu82HqZ24BgYGMGPGDLz77rtDfv7f//3fePvtt/E///M/KC0thVwux8qVK2Eymbh1NmzYgKqqKhQWFmL37t04dOgQHnzwQW/twoR0tXwBgFWrVrmcQ19++aXL5zRfPK+4uBgFBQU4duwYCgsLYbVasWLFCgwMDHDrXO36ZbfbcdNNN8FiseDo0aP45JNP8PHHH+PZZ5/1xS5NXIRncnNzSUFBAffabreTmJgY8vLLL/swVZPLc889R2bMmDHkZ319fUQkEpEdO3Zw750/f54AICUlJV5K4eQEgOzcuZN77XA4SFRUFPnzn//MvdfX10ckEgn58ssvCSGEnDt3jgAgJ06c4NbZs2cPYRiGtLa2ei3tE9kv84UQQu69915yyy23DPsdmi/eodVqCQBSXFxMCBnZ9evHH38kLMsSjUbDrfP++++ToKAgYjabvbsDExivSj4sFgvKysqwbNky7j2WZbFs2TKUlJT4MGWTT3V1NWJiYpCSkoINGzagqakJAFBWVgar1eqSR5mZmUhISKB55GX19fXQaDQueaFUKpGXl8flRUlJCVQqFebOncuts2zZMrAsi9LSUq+neTI5ePAgIiIiMGXKFPz+979Hd3c39xnNF+/Q6XQAgJCQEAAju36VlJRg+vTpiIyM5NZZuXIl+vv7UVVV5cXUT2y8Cj66urpgt9tdMh0AIiMjodFofJSqyScvLw8ff/wx9u7di/fffx/19fVYtGgR9Ho9NBoNxGIxVCqVy3doHnmf8+99pfNFo9EgIiLC5XOhUIiQkBCaX+No1apV+PTTT1FUVIRXX30VxcXFWL16Nex2OwCaL97gcDiwceNGLFiwANnZ2QAwouuXRqMZ8pxyfkZ5htDXCaD4Z/Xq1dz/c3JykJeXh8TERPzjH/+ATCbzYcooyj+sW7eO+//06dORk5OD1NRUHDx4EDfeeKMPUzZ5FBQUoLKy0qW9GsUfvCr5CAsLg0AgcGt53NHRgaioKB+lilKpVMjIyEBNTQ2ioqJgsVjQ19fnsg7NI+9z/r2vdL5ERUW5Nda22Wzo6emh+eVFKSkpCAsLQ01NDQCaL+PtkUcewe7du/HTTz8hLi6Oe38k16+oqKghzynnZ5Rn8Cr4EIvFmDNnDoqKirj3HA4HioqKkJ+f78OUTW4GgwG1tbWIjo7GnDlzIBKJXPJIrVajqamJ5pGXJScnIyoqyiUv+vv7UVpayuVFfn4++vr6UFZWxq1z4MABOBwO5OXleT3Nk1VLSwu6u7sRHR0NgObLeCGE4JFHHsHOnTtx4MABJCcnu3w+kutXfn4+zp496xIcFhYWIigoCFlZWd7ZkcnA1y1ef2nbtm1EIpGQjz/+mJw7d448+OCDRKVSubQ8psbX448/Tg4ePEjq6+vJkSNHyLJly0hYWBjRarWEEEIeeughkpCQQA4cOEBOnjxJ8vPzSX5+vo9TPTHp9XpSXl5OysvLCQDyl7/8hZSXl5PGxkZCCCGvvPIKUalU5NtvvyUVFRXklltuIcnJycRoNHLbWLVqFZk1axYpLS0lhw8fJunp6eTuu+/21S5NCFfKF71eT/7whz+QkpISUl9fT/bv309mz55N0tPTiclk4rZB88Xzfv/73xOlUkkOHjxI2tvbuWVwcJBb52rXL5vNRrKzs8mKFSvI6dOnyd69e0l4eDjZsmWLL3ZpwuJd8EEIIX/9619JQkICEYvFJDc3lxw7dszXSZpU7rrrLhIdHU3EYjGJjY0ld911F6mpqeE+NxqN5OGHHybBwcEkICCArF27lrS3t/swxRPXTz/9RAC4Lffeey8h5FJ322eeeYZERkYSiURCbrzxRqJWq1220d3dTe6++24SGBhIgoKCyH333Uf0er0P9mbiuFK+DA4OkhUrVpDw8HAiEolIYmIi+dd//Ve3ByiaL543VJ4AIFu3buXWGcn1q6GhgaxevZrIZDISFhZGHn/8cWK1Wr28NxMbQwgh3i5toSiKoihq8uJVmw+KoiiKoiY+GnxQFEVRFOVVNPigKIqiKMqraPBBURRFUZRX0eCDoiiKoiivosEHRVEURVFeRYMPiqIoiqK8igYfFEVRFEV5FQ0+KIqiKIryKhp8UBRFURTlVTT4oCiKoijKq/4/x2OUh5CfczcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GroundTruth: 7   2   1   0   4   1   4   9  \n",
      "Euclidean:   7   2   1   0   4   1   4   9  \n",
      "Hyperbolic:  7   2   1   0   4   1   4   9  \n"
     ]
    }
   ],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "flatten_images = torch.flatten(images, start_dim=1).to(device)\n",
    "# ======================================\n",
    "outputs = euclidean_net(flatten_images)\n",
    "_, euclidean_predicted = torch.max(outputs, 1)\n",
    "# ======================================\n",
    "hyp_inputs = manifold.expmap0(flatten_images)\n",
    "outputs = hyperbolic_net(hyp_inputs, manifold)\n",
    "_, hyperbolic_predicted = torch.max(outputs, 1)\n",
    "\n",
    "# print images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth:', ' '.join(f'{classes_num[labels[j]]:3s}' for j in range(8)))\n",
    "print('Euclidean:  ', ' '.join(f'{classes_num[euclidean_predicted[j]]:3s}' for j in range(8)))\n",
    "print('Hyperbolic: ', ' '.join(f'{classes_num[hyperbolic_predicted[j]]:3s}' for j in range(8)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the Euclidean network on the 10000 test images: 97 %\n",
      "Accuracy of the Hyperbolic network on the 10000 test images: 97 %\n"
     ]
    }
   ],
   "source": [
    "# Test dataset - downloading and loading the testing dataset.\n",
    "testset = torchvision.datasets.MNIST(root=DATA_PATH, train=False, download=True, transform=transform)\n",
    "testset_small, _ = torch.utils.data.random_split(main_trainset, [30000, 30000], generator=torch.Generator().manual_seed(42))\n",
    "\n",
    "# Create dataloaders for the train, val and test sets\n",
    "batch_size = 8\n",
    "testloader = torch.utils.data.DataLoader(testset_small, batch_size=batch_size, shuffle=False, num_workers=2)\n",
    "# testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size, shuffle=False, num_workers=2)\n",
    "\n",
    "euclidean_correct = 0\n",
    "hyperbolic_correct = 0\n",
    "total = 0\n",
    "# since we're not training, we don't need to calculate the gradients for our outputs\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        images = images.to(device)\n",
    "        labels = labels.to(device)\n",
    "        # calculate outputs by running images through the network\n",
    "        flatten_images = torch.flatten(images, start_dim=1)\n",
    "\n",
    "        euclidean_outputs = euclidean_net(flatten_images)\n",
    "        # ======================================\n",
    "        hyp_inputs = manifold.expmap0(flatten_images)\n",
    "        hyperbolic_outputs = hyperbolic_net(hyp_inputs, manifold)\n",
    "\n",
    "        # the class with the highest energy is what we choose as prediction\n",
    "        _, euclidean_predicted = torch.max(euclidean_outputs.data, 1)\n",
    "        _, hyperbolic_predicted = torch.max(hyperbolic_outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "\n",
    "        euclidean_correct += (euclidean_predicted == labels).sum().item()\n",
    "        hyperbolic_correct += (hyperbolic_predicted == labels).sum().item()\n",
    "\n",
    "print(f'Accuracy of the Euclidean network on the 10000 test images: {100 * euclidean_correct // total} %')\n",
    "print(f'Accuracy of the Hyperbolic network on the 10000 test images: {100 * hyperbolic_correct // total} %')"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.13"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "002bd74778cb4b8da294b4d5f4923e5a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "052d04deed0b4a0eb3fc953ab255e5e4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "082b385bad4d4e7281a07f77f0cc9cd4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "08cc32263dfa421fa7040d3ba8d22bd8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "10ba0efce9e5430c902cdcf56bab09e4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5331b18b991f46f1815942d08c267b44",
      "placeholder": "​",
      "style": "IPY_MODEL_08cc32263dfa421fa7040d3ba8d22bd8",
      "value": " 1648877/1648877 [00:00&lt;00:00, 10480297.85it/s]"
     }
    },
    "16979721c7444c52a282b318160c9586": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1e655534822245e8adb36db4cb115d13": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d02055dd894c442280f796d130869884",
      "max": 1648877,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_de94640096be496399872f2e94bec4ad",
      "value": 1648877
     }
    },
    "2153a2fbcd98499cafadc5b413cc25f7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2192a5dd584a487a903a60b7837f9d65": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "51fae820bd914df5b65bbb2678cc05f8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5331b18b991f46f1815942d08c267b44": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "550154eb777248e590bb8db09ff7a90c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e490e5a6fbea4c87839b4bdfadc8c52e",
       "IPY_MODEL_e010dc5beaf6408f91b22cbac009a395",
       "IPY_MODEL_9d4dd00ba16b403cb233336aab50b80c"
      ],
      "layout": "IPY_MODEL_51fae820bd914df5b65bbb2678cc05f8"
     }
    },
    "5c247e41508f4bdb8b7c768bf093accb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_082b385bad4d4e7281a07f77f0cc9cd4",
      "placeholder": "​",
      "style": "IPY_MODEL_2153a2fbcd98499cafadc5b413cc25f7",
      "value": "100%"
     }
    },
    "62017fd3a3014c04ad6294d363771cf0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "62b3030b82724d72b2bccc65526778da": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "637a7991c79b49f9928259e386c31e7d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_97074fecae464f3f87e907c4f0cf1884",
      "placeholder": "​",
      "style": "IPY_MODEL_052d04deed0b4a0eb3fc953ab255e5e4",
      "value": " 9912422/9912422 [00:00&lt;00:00, 7849001.15it/s]"
     }
    },
    "6a98047e1ab944899984c2b1d18d78f4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9e8e11e458054415a57572a6665fc8a8",
      "placeholder": "​",
      "style": "IPY_MODEL_8d284174e0d64be6a57e9ab5d78eda11",
      "value": " 28881/28881 [00:00&lt;00:00, 790527.52it/s]"
     }
    },
    "6f7b6a2d42a64e0aa3a285ccb9a7ab43": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_9a862696188a46d1aa314af61d6574d6",
       "IPY_MODEL_b581f32916574ee6966434311422bf99",
       "IPY_MODEL_6a98047e1ab944899984c2b1d18d78f4"
      ],
      "layout": "IPY_MODEL_a596527976494de8b123c198ed6ad481"
     }
    },
    "801007438163403591daf4218a0825cc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_16979721c7444c52a282b318160c9586",
      "placeholder": "​",
      "style": "IPY_MODEL_002bd74778cb4b8da294b4d5f4923e5a",
      "value": "100%"
     }
    },
    "816d67b923bd4a53b427ee385943b4df": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "89b8e5dd0da84f96965e649613e14c54": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8d284174e0d64be6a57e9ab5d78eda11": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "97074fecae464f3f87e907c4f0cf1884": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "995d015dcf3b452e85742c5eea660064": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_801007438163403591daf4218a0825cc",
       "IPY_MODEL_c9124614852d46d9a111bec0ec03e509",
       "IPY_MODEL_637a7991c79b49f9928259e386c31e7d"
      ],
      "layout": "IPY_MODEL_62017fd3a3014c04ad6294d363771cf0"
     }
    },
    "9a862696188a46d1aa314af61d6574d6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bd0f6652f5dd4f799cdc5c308bb21bf5",
      "placeholder": "​",
      "style": "IPY_MODEL_dd67739ea09641b396376108d1d0208b",
      "value": "100%"
     }
    },
    "9d4dd00ba16b403cb233336aab50b80c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_daecebdd1d6648da9a1fb55ab0c476de",
      "placeholder": "​",
      "style": "IPY_MODEL_ed605bbbb43b4fdf9bbf8daa08093341",
      "value": " 4542/4542 [00:00&lt;00:00, 85703.17it/s]"
     }
    },
    "9e8e11e458054415a57572a6665fc8a8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a596527976494de8b123c198ed6ad481": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b581f32916574ee6966434311422bf99": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cf7ba4a0d470417e9f8254d3d6840ea5",
      "max": 28881,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2192a5dd584a487a903a60b7837f9d65",
      "value": 28881
     }
    },
    "bd0f6652f5dd4f799cdc5c308bb21bf5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c9124614852d46d9a111bec0ec03e509": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d79cd089fe6540b091b55c677b62967e",
      "max": 9912422,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d1051e8fde8f4adf8718b27ddbf7c62a",
      "value": 9912422
     }
    },
    "cf7ba4a0d470417e9f8254d3d6840ea5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d02055dd894c442280f796d130869884": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d1051e8fde8f4adf8718b27ddbf7c62a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d79cd089fe6540b091b55c677b62967e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "daecebdd1d6648da9a1fb55ab0c476de": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dd67739ea09641b396376108d1d0208b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "de94640096be496399872f2e94bec4ad": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e010dc5beaf6408f91b22cbac009a395": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_816d67b923bd4a53b427ee385943b4df",
      "max": 4542,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_62b3030b82724d72b2bccc65526778da",
      "value": 4542
     }
    },
    "e490e5a6fbea4c87839b4bdfadc8c52e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f6aa342888e54445b4447a9fff11f9dc",
      "placeholder": "​",
      "style": "IPY_MODEL_89b8e5dd0da84f96965e649613e14c54",
      "value": "100%"
     }
    },
    "ed605bbbb43b4fdf9bbf8daa08093341": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "eecdc0d6ec0e499cbdbce64783ed1123": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5c247e41508f4bdb8b7c768bf093accb",
       "IPY_MODEL_1e655534822245e8adb36db4cb115d13",
       "IPY_MODEL_10ba0efce9e5430c902cdcf56bab09e4"
      ],
      "layout": "IPY_MODEL_f243551123fc4dd6a05a9486200614a5"
     }
    },
    "f243551123fc4dd6a05a9486200614a5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f6aa342888e54445b4447a9fff11f9dc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
